[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:30 dist-test-slave-1nl6 krb5kdc[8261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:30 dist-test-slave-1nl6 krb5kdc[8261](info): set up 2 sockets
Aug 08 16:56:30 dist-test-slave-1nl6 krb5kdc[8261](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 16:56:31.658094 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:56:31 dist-test-slave-1nl6 krb5kdc[8261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:56:31.778095 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:38231
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:44019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:38231
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:31.862808 8278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:31.863102 8278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:31.863152 8278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:31.864289 8278 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:31.864356 8278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:31.864383 8278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:31.864414 8278 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:31.864439 8278 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:31.865955 8278 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:44019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:38231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:38231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:31.866250 8278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:31.866487 8278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:31.871922 8287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:31.871923 8286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:31.871953 8291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:31.872628 8278 server_base.cc:1034] running on GCE node
I20260808 16:56:31.872926 8278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:31.873306 8278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:31.874498 8278 hybrid_clock.cc:648] HybridClock initialized: now 1786208191874431 us; error 82 us; skew 500 ppm
Aug 08 16:56:31 dist-test-slave-1nl6 krb5kdc[8261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208191, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:31.877952 8278 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:56:31.878453 8278 webserver.cc:460] Webserver started at http://127.8.14.126:39869/ using document root <none> and password file <none>
I20260808 16:56:31.878736 8278 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:31.878818 8278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:31.879022 8278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:31.880259 8278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "690a3d6808474d09a45d4f471a10d6f9"
format_stamp: "Formatted at 2026-08-08 16:56:31 on dist-test-slave-1nl6"
server_key: "4d63f5bcfbd47daa2c850de77d221f37"
server_key_iv: "1f18d9a6a0f0a53ad02013b562fa77b4"
server_key_version: "encryptionkey@0"
I20260808 16:56:31.880682 8278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "690a3d6808474d09a45d4f471a10d6f9"
format_stamp: "Formatted at 2026-08-08 16:56:31 on dist-test-slave-1nl6"
server_key: "4d63f5bcfbd47daa2c850de77d221f37"
server_key_iv: "1f18d9a6a0f0a53ad02013b562fa77b4"
server_key_version: "encryptionkey@0"
I20260808 16:56:31.887305 8278 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.001s
I20260808 16:56:31.888237 8302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:31.888417 8278 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:31.888556 8278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "690a3d6808474d09a45d4f471a10d6f9"
format_stamp: "Formatted at 2026-08-08 16:56:31 on dist-test-slave-1nl6"
server_key: "4d63f5bcfbd47daa2c850de77d221f37"
server_key_iv: "1f18d9a6a0f0a53ad02013b562fa77b4"
server_key_version: "encryptionkey@0"
I20260808 16:56:31.888664 8278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:31.915911 8278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:31.919157 8278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:31.919355 8278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:31.923590 8278 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:38231
I20260808 16:56:31.923667 8376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:38231 every 8 connection(s)
I20260808 16:56:31.924027 8278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:31.925194 8377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:31.928829 8377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9: Bootstrap starting.
I20260808 16:56:31.929594 8377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:31.929948 8377 log.cc:824] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:31.930795 8377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9: No bootstrap required, opened a new log
I20260808 16:56:31.933349 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 8278
I20260808 16:56:31.933471 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:56:31.933525 8377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "690a3d6808474d09a45d4f471a10d6f9" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 38231 } }
I20260808 16:56:31.933722 8377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:31.933768 8377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 690a3d6808474d09a45d4f471a10d6f9, State: Initialized, Role: FOLLOWER
I20260808 16:56:31.933938 8377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "690a3d6808474d09a45d4f471a10d6f9" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 38231 } }
I20260808 16:56:31.934021 8377 raft_consensus.cc:388] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:31.934062 8377 raft_consensus.cc:482] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:31.934105 8377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:31.934283 8249 external_mini_cluster.cc:1428] Setting key 6749df96d1fe578006af27cd5708351d
I20260808 16:56:31.934872 8377 raft_consensus.cc:504] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "690a3d6808474d09a45d4f471a10d6f9" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 38231 } }
I20260808 16:56:31.935021 8377 leader_election.cc:302] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 690a3d6808474d09a45d4f471a10d6f9; no voters:
I20260808 16:56:31.935403 8377 leader_election.cc:288] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:31.935570 8387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:31.935680 8377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:31.935693 8387 raft_consensus.cc:686] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [term 1 LEADER]: Becoming Leader. State: Replica: 690a3d6808474d09a45d4f471a10d6f9, State: Running, Role: LEADER
I20260808 16:56:31.935796 8387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "690a3d6808474d09a45d4f471a10d6f9" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 38231 } }
I20260808 16:56:31.936120 8387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "690a3d6808474d09a45d4f471a10d6f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "690a3d6808474d09a45d4f471a10d6f9" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 38231 } } }
I20260808 16:56:31.936214 8387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:31.936368 8387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 690a3d6808474d09a45d4f471a10d6f9. Latest consensus state: current_term: 1 leader_uuid: "690a3d6808474d09a45d4f471a10d6f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "690a3d6808474d09a45d4f471a10d6f9" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 38231 } } }
I20260808 16:56:31.936434 8387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9 [sys.catalog]: This master's current role is: LEADER
W20260808 16:56:31.937884 8406 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:56:31.937987 8406 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:56:31.938095 8394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:31.938314 8394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:31.939874 8394 catalog_manager.cc:1347] Generated new cluster ID: f9ed007c8cc948dba739468751d2fc7d
I20260808 16:56:31.939934 8394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:56:31 dist-test-slave-1nl6 krb5kdc[8261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:31.953388 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:31.935175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44206 (local address 127.8.14.126:38231)
0808 16:56:31.935373 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:31.935382 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:31.935729 (+ 347us) server_negotiation.cc:407] Connection header received
0808 16:56:31.938370 (+ 2641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:31.938376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:31.938430 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:31.938569 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:31.940505 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:31.941838 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:31.943413 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:31.944017 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:31.944098 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:31.948931 (+ 4833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:31.948935 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:31.948935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:31.948947 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:31.951159 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:31.951680 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:31.951684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:31.951684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:31.951804 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:31.952131 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:31.952132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:31.952132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:31.952444 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:31.952588 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:31.953007 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 16:56:31.953148 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
I20260808 16:56:31.962033 8394 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:31.962878 8394 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:31.968717 8394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 690a3d6808474d09a45d4f471a10d6f9: Generated new TSK 0
I20260808 16:56:31.968921 8394 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 16:56:32.001351 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:38231
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:44019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:32.084971 8411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:32.085196 8411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:32.085235 8411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:32.086364 8411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:32.086432 8411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:32.086478 8411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:56:32.088021 8411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:44019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:38231
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:32.088289 8411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:32.088521 8411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:32.093662 8417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:32.093629 8411 server_base.cc:1034] running on GCE node
W20260808 16:56:32.093649 8416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:32.093680 8420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:32.094050 8411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:32.094288 8411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:32.095448 8411 hybrid_clock.cc:648] HybridClock initialized: now 1786208192095439 us; error 40 us; skew 500 ppm
Aug 08 16:56:32 dist-test-slave-1nl6 krb5kdc[8261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208192, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:32.097826 8411 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:56:32.098280 8411 webserver.cc:460] Webserver started at http://127.8.14.65:43235/ using document root <none> and password file <none>
I20260808 16:56:32.098520 8411 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:32.098590 8411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:32.098729 8411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:32.099814 8411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "e6b74aa2098740dd92eb0dd08be3d7a9"
format_stamp: "Formatted at 2026-08-08 16:56:32 on dist-test-slave-1nl6"
server_key: "ad7a7cefc3ce24d2eb0b37a9e1484be4"
server_key_iv: "04a89a710e1da1c7c25ebde771757402"
server_key_version: "encryptionkey@0"
I20260808 16:56:32.100171 8411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "e6b74aa2098740dd92eb0dd08be3d7a9"
format_stamp: "Formatted at 2026-08-08 16:56:32 on dist-test-slave-1nl6"
server_key: "ad7a7cefc3ce24d2eb0b37a9e1484be4"
server_key_iv: "04a89a710e1da1c7c25ebde771757402"
server_key_version: "encryptionkey@0"
I20260808 16:56:32.101487 8411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:56:32.102229 8429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:32.102412 8411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:32.102511 8411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "e6b74aa2098740dd92eb0dd08be3d7a9"
format_stamp: "Formatted at 2026-08-08 16:56:32 on dist-test-slave-1nl6"
server_key: "ad7a7cefc3ce24d2eb0b37a9e1484be4"
server_key_iv: "04a89a710e1da1c7c25ebde771757402"
server_key_version: "encryptionkey@0"
I20260808 16:56:32.102598 8411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:32.115576 8411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:32.117369 8411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:32.117523 8411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:32.117739 8411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:32.118042 8411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:32.118098 8411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:32.118139 8411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:32.118165 8411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:32.124220 8411 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:37631
I20260808 16:56:32.124274 8597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:37631 every 8 connection(s)
I20260808 16:56:32.124567 8411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:56:32.127259 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 8411
I20260808 16:56:32.127372 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:56:32.127521 8249 external_mini_cluster.cc:1428] Setting key 875056c5e9e40ef8c1211d83cb6261ce
Aug 08 16:56:32 dist-test-slave-1nl6 krb5kdc[8261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208192, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:32.134426 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:32.125338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:49229 (local address 127.8.14.126:38231)
0808 16:56:32.125422 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:32.125425 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:32.125585 (+ 160us) server_negotiation.cc:407] Connection header received
0808 16:56:32.126040 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:32.126041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:32.126055 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:32.126102 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:32.127105 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:32.127723 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:32.128832 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:32.129055 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:32.129144 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:32.131373 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:32.131374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:32.131375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:32.131383 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:32.132991 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:32.133434 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:32.133435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:32.133436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:32.133500 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:32.133755 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:32.133756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:32.133756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:32.133879 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:32.133965 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:32.134235 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 16:56:32.134335 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 16:56:32.134610 8600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:32.125284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:38231 (local address 127.8.14.65:49229)
0808 16:56:32.125469 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:32.125503 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:32.125959 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:32.126162 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:32.126164 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:32.126805 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:32.127015 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:32.127020 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:32.127821 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:32.127823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:32.128722 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:32.128725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:32.129157 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:32.129158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:32.129294 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:32.129861 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:32.129872 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:32.131280 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:32.133100 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:32.133107 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:32.133110 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:32.133339 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:32.133582 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:32.133583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:32.133583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:32.133685 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:32.133955 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:32.133961 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:32.134098 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 16:56:32.134199 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 16:56:32.134363 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
I20260808 16:56:32.135098 8598 heartbeater.cc:347] Connected to a master server at 127.8.14.126:38231
I20260808 16:56:32.135196 8598 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:32.135371 8598 heartbeater.cc:511] Master 127.8.14.126:38231 requested a full tablet report, sending...
I20260808 16:56:32.135980 8327 ts_manager.cc:194] Registered new tserver with Master: e6b74aa2098740dd92eb0dd08be3d7a9 (127.8.14.65:37631)
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
I20260808 16:56:32.136780 8327 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:49229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 16:56:32.157397 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:38231
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:44019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:32.237876 8604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:32.238095 8604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:32.238133 8604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:32.239272 8604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:32.239341 8604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:32.239387 8604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:56:32.240814 8604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:44019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:38231
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:32.241070 8604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:32.241302 8604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:56:32.246222 8604 server_base.cc:1034] running on GCE node
W20260808 16:56:32.246186 8616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:32.246186 8611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:32.246176 8612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:32.246698 8604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:32.246978 8604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:32.248155 8604 hybrid_clock.cc:648] HybridClock initialized: now 1786208192248143 us; error 26 us; skew 500 ppm
Aug 08 16:56:32 dist-test-slave-1nl6 krb5kdc[8261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208192, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:32.253168 8604 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:56:32.253583 8604 webserver.cc:460] Webserver started at http://127.8.14.66:35801/ using document root <none> and password file <none>
I20260808 16:56:32.253801 8604 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:32.253868 8604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:32.254009 8604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:32.255066 8604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "22bfd69209964b7a9a00e313ea0e9943"
format_stamp: "Formatted at 2026-08-08 16:56:32 on dist-test-slave-1nl6"
server_key: "f51ac05683bf03e1cab200f4957ed8a7"
server_key_iv: "57b3b9c3f2725cc5fbaf4d9bcdbf8a39"
server_key_version: "encryptionkey@0"
I20260808 16:56:32.255427 8604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "22bfd69209964b7a9a00e313ea0e9943"
format_stamp: "Formatted at 2026-08-08 16:56:32 on dist-test-slave-1nl6"
server_key: "f51ac05683bf03e1cab200f4957ed8a7"
server_key_iv: "57b3b9c3f2725cc5fbaf4d9bcdbf8a39"
server_key_version: "encryptionkey@0"
I20260808 16:56:32.256773 8604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:56:32.257455 8626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:32.257619 8604 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:56:32.257714 8604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "22bfd69209964b7a9a00e313ea0e9943"
format_stamp: "Formatted at 2026-08-08 16:56:32 on dist-test-slave-1nl6"
server_key: "f51ac05683bf03e1cab200f4957ed8a7"
server_key_iv: "57b3b9c3f2725cc5fbaf4d9bcdbf8a39"
server_key_version: "encryptionkey@0"
I20260808 16:56:32.257805 8604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:32.279898 8604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:32.281726 8604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:32.281894 8604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:32.282128 8604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:32.282472 8604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:32.282534 8604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:32.282577 8604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:32.282603 8604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:32.289758 8604 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:33225
I20260808 16:56:32.289801 8781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:33225 every 8 connection(s)
I20260808 16:56:32.290156 8604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:56:32.292681 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 8604
I20260808 16:56:32.292794 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:56:32.292973 8249 external_mini_cluster.cc:1428] Setting key df30ea7ca99529cbe0982adebf54f28d
Aug 08 16:56:32 dist-test-slave-1nl6 krb5kdc[8261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208192, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:32.300392 8784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:32.290957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:38231 (local address 127.8.14.66:41697)
0808 16:56:32.291128 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:32.291164 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:32.291703 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:32.291915 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:32.291918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:32.292548 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:32.292752 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:32.292757 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:32.293688 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:32.293689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:32.294493 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:32.294495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:32.295043 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:32.295044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:32.295145 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:32.295674 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:32.295682 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:32.297175 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:32.299074 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:32.299080 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:32.299081 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:32.299294 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:32.299522 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:32.299522 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:32.299523 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:32.299615 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:32.299920 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:32.299923 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:32.300033 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 16:56:32.300112 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 16:56:32.300227 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
I20260808 16:56:32.300391 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:32.290979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:41697 (local address 127.8.14.126:38231)
0808 16:56:32.291050 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:32.291053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:32.291274 (+ 221us) server_negotiation.cc:407] Connection header received
0808 16:56:32.291800 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:32.291800 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:32.291811 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:32.291863 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:32.292876 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:32.293600 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:32.294583 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:32.294964 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:32.295038 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:32.297266 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:32.297268 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:32.297268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:32.297276 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:32.298997 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:32.299377 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:32.299379 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:32.299379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:32.299447 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:32.299683 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:32.299684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:32.299685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:32.299848 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:32.299928 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:32.300164 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 16:56:32.300258 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 16:56:32.300918 8782 heartbeater.cc:347] Connected to a master server at 127.8.14.126:38231
I20260808 16:56:32.301003 8782 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:32.301203 8782 heartbeater.cc:511] Master 127.8.14.126:38231 requested a full tablet report, sending...
I20260808 16:56:32.301652 8327 ts_manager.cc:194] Registered new tserver with Master: 22bfd69209964b7a9a00e313ea0e9943 (127.8.14.66:33225)
I20260808 16:56:32.302182 8327 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:41697
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 16:56:32.319384 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:38231
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:44019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:32.400513 8787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:32.400712 8787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:32.400735 8787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:32.401809 8787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:32.401842 8787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:32.401867 8787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:56:32.403301 8787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:44019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:38231
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:32.403534 8787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:32.403753 8787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:32.408622 8795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:32.408614 8799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:32.408628 8797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:32.408860 8787 server_base.cc:1034] running on GCE node
I20260808 16:56:32.409045 8787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:32.409271 8787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:32.410414 8787 hybrid_clock.cc:648] HybridClock initialized: now 1786208192410402 us; error 26 us; skew 500 ppm
Aug 08 16:56:32 dist-test-slave-1nl6 krb5kdc[8261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208192, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:32.412885 8787 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:56:32.413322 8787 webserver.cc:460] Webserver started at http://127.8.14.67:33729/ using document root <none> and password file <none>
I20260808 16:56:32.413508 8787 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:32.413545 8787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:32.413676 8787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:32.414721 8787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "8aa84b4f432543ddbd1c6f03b8680ffc"
format_stamp: "Formatted at 2026-08-08 16:56:32 on dist-test-slave-1nl6"
server_key: "bf64021a6b523318076ece5ca3e73382"
server_key_iv: "316ce3e8030a8a759c288462d68b4a50"
server_key_version: "encryptionkey@0"
I20260808 16:56:32.415107 8787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "8aa84b4f432543ddbd1c6f03b8680ffc"
format_stamp: "Formatted at 2026-08-08 16:56:32 on dist-test-slave-1nl6"
server_key: "bf64021a6b523318076ece5ca3e73382"
server_key_iv: "316ce3e8030a8a759c288462d68b4a50"
server_key_version: "encryptionkey@0"
I20260808 16:56:32.416407 8787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:56:32.417132 8807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:32.417317 8787 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:56:32.417419 8787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "8aa84b4f432543ddbd1c6f03b8680ffc"
format_stamp: "Formatted at 2026-08-08 16:56:32 on dist-test-slave-1nl6"
server_key: "bf64021a6b523318076ece5ca3e73382"
server_key_iv: "316ce3e8030a8a759c288462d68b4a50"
server_key_version: "encryptionkey@0"
I20260808 16:56:32.417500 8787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:32.428243 8787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:32.429955 8787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:32.430110 8787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:32.430330 8787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:32.430637 8787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:32.430693 8787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:32.430733 8787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:32.430763 8787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:32.436928 8787 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:45301
I20260808 16:56:32.437000 8971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:45301 every 8 connection(s)
I20260808 16:56:32.437350 8787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
Aug 08 16:56:32 dist-test-slave-1nl6 krb5kdc[8261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208192, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:32.444011 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 8787
I20260808 16:56:32.444094 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:56:32.444272 8249 external_mini_cluster.cc:1428] Setting key 954e2830417819322d44e47689cd19a8
I20260808 16:56:32.446513 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:32.438029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:35483 (local address 127.8.14.126:38231)
0808 16:56:32.438070 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:32.438073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:32.438278 (+ 205us) server_negotiation.cc:407] Connection header received
0808 16:56:32.438747 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:32.438748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:32.438757 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:32.438815 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:32.439734 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:32.440210 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:32.441123 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:32.441302 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:32.441351 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:32.443510 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:32.443512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:32.443512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:32.443519 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:32.445194 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:32.445560 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:32.445561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:32.445561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:32.445618 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:32.445885 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:32.445886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:32.445886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:32.446002 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:32.446077 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:32.446346 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 16:56:32.446424 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:56:32.446696 8976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:32.437991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:38231 (local address 127.8.14.67:35483)
0808 16:56:32.438149 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:32.438189 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:32.438664 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:32.438879 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:32.438882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:32.439478 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:32.439655 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:32.439660 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:32.440290 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:32.440291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:32.441053 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:32.441055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:32.441373 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:32.441374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:32.441466 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:32.441912 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:32.441919 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:32.443410 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:32.445254 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:32.445261 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:32.445263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:32.445479 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:32.445692 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:32.445693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:32.445694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:32.445806 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:32.446069 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:32.446075 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:32.446210 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 16:56:32.446321 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 16:56:32.446475 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260808 16:56:32.447130 8973 heartbeater.cc:347] Connected to a master server at 127.8.14.126:38231
I20260808 16:56:32.447216 8973 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:32.447348 8973 heartbeater.cc:511] Master 127.8.14.126:38231 requested a full tablet report, sending...
I20260808 16:56:32.447685 8327 ts_manager.cc:194] Registered new tserver with Master: 8aa84b4f432543ddbd1c6f03b8680ffc (127.8.14.67:45301)
I20260808 16:56:32.448060 8327 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:35483
I20260808 16:56:32.456138 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 16:56:32 dist-test-slave-1nl6 krb5kdc[8261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208192, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 16:56:32 dist-test-slave-1nl6 krb5kdc[8261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208192, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.14.65@KRBTEST.COM
I20260808 16:56:32.479575 8985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:32.472314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48560 (local address 127.8.14.65:37631)
0808 16:56:32.472496 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:32.472501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:32.472511 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:56:32.472537 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:32.472539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:32.472553 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:32.472618 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:32.473130 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:32.473820 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:32.474507 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:32.474880 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:32.474998 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:32.477078 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:32.477086 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:32.477087 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:32.477093 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:32.478454 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:32.478805 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:32.478808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:32.478808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:32.478862 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:32.479077 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:32.479078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:32.479078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:32.479209 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:32.479313 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:32.479382 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 16:56:32.479445 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":89,"threads_started":1}
W20260808 16:56:32.479990 8471 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48560
Aug 08 16:56:32 dist-test-slave-1nl6 krb5kdc[8261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 16:56:32 dist-test-slave-1nl6 krb5kdc[8261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.65@KRBTEST.COM
I20260808 16:56:32.502485 8985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:32.495808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48566 (local address 127.8.14.65:37631)
0808 16:56:32.495865 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:32.495867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:32.495926 (+ 59us) server_negotiation.cc:407] Connection header received
0808 16:56:32.495986 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:32.495987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:32.495999 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:32.496047 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:32.496617 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:32.497096 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:32.497772 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:32.497973 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:32.498028 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:32.500239 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:32.500241 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:32.500241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:32.500247 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:32.501611 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:32.501890 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:32.501890 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:32.501891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:32.501940 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:32.502086 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:32.502086 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:32.502086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:32.502165 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:32.502246 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:32.502317 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 16:56:32.502376 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 16:56:32.503054 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 8411
I20260808 16:56:32.514704 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 8604
I20260808 16:56:32.521090 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 8787
I20260808 16:56:32.528898 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 8278
2026-08-08T16:56:32Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2100 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:32 dist-test-slave-1nl6 krb5kdc[8997](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:32 dist-test-slave-1nl6 krb5kdc[8997](info): set up 2 sockets
Aug 08 16:56:32 dist-test-slave-1nl6 krb5kdc[8997](info): commencing operation
krb5kdc: starting...
W20260808 16:56:33.743093 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:56:33 dist-test-slave-1nl6 krb5kdc[8997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:56:33.859988 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:43709
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:41011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:43709
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:33.940833 9016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:33.941073 9016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:33.941111 9016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:33.942225 9016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:33.942302 9016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:33.942328 9016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:33.942354 9016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:33.942381 9016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:33.943818 9016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:43709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:43709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:33.944094 9016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:33.944332 9016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:33.949101 9029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:33.949110 9024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:33.949324 9027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:33.949332 9016 server_base.cc:1034] running on GCE node
I20260808 16:56:33.949621 9016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:33.949936 9016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:33.951076 9016 hybrid_clock.cc:648] HybridClock initialized: now 1786208193951060 us; error 34 us; skew 500 ppm
Aug 08 16:56:33 dist-test-slave-1nl6 krb5kdc[8997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208193, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:33.953725 9016 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:56:33.954273 9016 webserver.cc:460] Webserver started at http://127.8.14.126:41705/ using document root <none> and password file <none>
I20260808 16:56:33.954520 9016 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:33.954586 9016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:33.954725 9016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:33.956158 9016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "637cc490a3f54273b7f82d3adbe93364"
format_stamp: "Formatted at 2026-08-08 16:56:33 on dist-test-slave-1nl6"
server_key: "c8addfd017b8e33ab403e53589d2e5ef"
server_key_iv: "34d0b2b08a3cd808e12b911d7024cd24"
server_key_version: "encryptionkey@0"
I20260808 16:56:33.956629 9016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "637cc490a3f54273b7f82d3adbe93364"
format_stamp: "Formatted at 2026-08-08 16:56:33 on dist-test-slave-1nl6"
server_key: "c8addfd017b8e33ab403e53589d2e5ef"
server_key_iv: "34d0b2b08a3cd808e12b911d7024cd24"
server_key_version: "encryptionkey@0"
I20260808 16:56:33.958597 9016 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 16:56:33.959787 9042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:33.964183 9016 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260808 16:56:33.964308 9016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "637cc490a3f54273b7f82d3adbe93364"
format_stamp: "Formatted at 2026-08-08 16:56:33 on dist-test-slave-1nl6"
server_key: "c8addfd017b8e33ab403e53589d2e5ef"
server_key_iv: "34d0b2b08a3cd808e12b911d7024cd24"
server_key_version: "encryptionkey@0"
I20260808 16:56:33.964396 9016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:33.979900 9016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:33.982242 9016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:33.982383 9016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:33.987703 9016 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:43709
I20260808 16:56:33.987736 9117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:43709 every 8 connection(s)
I20260808 16:56:33.988190 9016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:33.989398 9118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:33.993775 9118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364: Bootstrap starting.
I20260808 16:56:33.994552 9118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:33.994715 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 9016
I20260808 16:56:33.994807 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:56:33.994937 9118 log.cc:824] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:33.994998 8249 external_mini_cluster.cc:1428] Setting key e287f5fa3d92c9109e29cf1fa3f8cfc5
I20260808 16:56:33.995715 9118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364: No bootstrap required, opened a new log
I20260808 16:56:33.998387 9118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "637cc490a3f54273b7f82d3adbe93364" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 43709 } }
I20260808 16:56:33.998613 9118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:33.998663 9118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 637cc490a3f54273b7f82d3adbe93364, State: Initialized, Role: FOLLOWER
I20260808 16:56:33.998832 9118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "637cc490a3f54273b7f82d3adbe93364" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 43709 } }
I20260808 16:56:33.998956 9118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:33.998993 9118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:33.999037 9118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:33.999534 9118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "637cc490a3f54273b7f82d3adbe93364" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 43709 } }
I20260808 16:56:33.999610 9118 leader_election.cc:302] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 637cc490a3f54273b7f82d3adbe93364; no voters:
I20260808 16:56:33.999774 9118 leader_election.cc:288] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:33.999809 9124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:33.999919 9124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [term 1 LEADER]: Becoming Leader. State: Replica: 637cc490a3f54273b7f82d3adbe93364, State: Running, Role: LEADER
I20260808 16:56:33.999991 9124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "637cc490a3f54273b7f82d3adbe93364" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 43709 } }
I20260808 16:56:34.000029 9118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:34.000324 9124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 637cc490a3f54273b7f82d3adbe93364. Latest consensus state: current_term: 1 leader_uuid: "637cc490a3f54273b7f82d3adbe93364" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "637cc490a3f54273b7f82d3adbe93364" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 43709 } } }
I20260808 16:56:34.000392 9124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:34.000334 9125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "637cc490a3f54273b7f82d3adbe93364" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "637cc490a3f54273b7f82d3adbe93364" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 43709 } } }
I20260808 16:56:34.000437 9125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364 [sys.catalog]: This master's current role is: LEADER
W20260808 16:56:34.001806 9150 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:56:34.002635 9150 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:56:34.002720 9134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:34.002890 9134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:34.004702 9134 catalog_manager.cc:1347] Generated new cluster ID: bd7f9b9b8fbc445386ee938285f227d2
I20260808 16:56:34.004774 9134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:56:33 dist-test-slave-1nl6 krb5kdc[8997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:34.010340 9123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:33.995554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40870 (local address 127.8.14.126:43709)
0808 16:56:33.995899 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:33.995911 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:33.995945 (+ 34us) server_negotiation.cc:407] Connection header received
0808 16:56:33.998697 (+ 2752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:33.998707 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:33.998770 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:33.998946 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:33.999878 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.001093 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.002366 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.002952 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.003070 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.005938 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:34.005943 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:34.005943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:34.005956 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:34.008714 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:34.009096 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:34.009100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:34.009100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:34.009215 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:34.009426 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:34.009427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:34.009428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:34.009758 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:34.009919 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:34.009999 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 16:56:34.010140 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":71,"threads_started":1}
I20260808 16:56:34.015527 9134 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:34.016089 9134 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:34.025882 9134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 637cc490a3f54273b7f82d3adbe93364: Generated new TSK 0
I20260808 16:56:34.026058 9134 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 16:56:34.057510 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:43709
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:41011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:34.170743 9162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:34.171002 9162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:34.171030 9162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:34.172199 9162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:34.172247 9162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:34.172303 9162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:56:34.173769 9162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:43709
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:34.174023 9162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:34.174260 9162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:34.179358 9172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:34.179394 9168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:34.179561 9162 server_base.cc:1034] running on GCE node
W20260808 16:56:34.179358 9169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:34.179862 9162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:34.180063 9162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:34.181227 9162 hybrid_clock.cc:648] HybridClock initialized: now 1786208194181210 us; error 45 us; skew 500 ppm
Aug 08 16:56:34 dist-test-slave-1nl6 krb5kdc[8997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208194, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:34.183718 9162 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:56:34.184167 9162 webserver.cc:460] Webserver started at http://127.8.14.65:33611/ using document root <none> and password file <none>
I20260808 16:56:34.184398 9162 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:34.184444 9162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:34.184589 9162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:34.185618 9162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "0b6927589e7043d382491248f3449d88"
format_stamp: "Formatted at 2026-08-08 16:56:34 on dist-test-slave-1nl6"
server_key: "fbdffa7ccba07f5d820c5f236bc51483"
server_key_iv: "64c1ae89c4922220f1e51b217ee6ad25"
server_key_version: "encryptionkey@0"
I20260808 16:56:34.185973 9162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "0b6927589e7043d382491248f3449d88"
format_stamp: "Formatted at 2026-08-08 16:56:34 on dist-test-slave-1nl6"
server_key: "fbdffa7ccba07f5d820c5f236bc51483"
server_key_iv: "64c1ae89c4922220f1e51b217ee6ad25"
server_key_version: "encryptionkey@0"
I20260808 16:56:34.187391 9162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:34.188103 9180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:34.188287 9162 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:34.188395 9162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "0b6927589e7043d382491248f3449d88"
format_stamp: "Formatted at 2026-08-08 16:56:34 on dist-test-slave-1nl6"
server_key: "fbdffa7ccba07f5d820c5f236bc51483"
server_key_iv: "64c1ae89c4922220f1e51b217ee6ad25"
server_key_version: "encryptionkey@0"
I20260808 16:56:34.188469 9162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:34.208863 9162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:34.210726 9162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:34.210916 9162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:34.211189 9162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:34.211516 9162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:34.211575 9162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:34.211616 9162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:34.211642 9162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:34.217681 9162 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:46067
I20260808 16:56:34.217734 9345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:46067 every 8 connection(s)
I20260808 16:56:34.218065 9162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
Aug 08 16:56:34 dist-test-slave-1nl6 krb5kdc[8997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208194, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:34.224820 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 9162
I20260808 16:56:34.224920 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:56:34.225101 8249 external_mini_cluster.cc:1428] Setting key d1f5d056e18a5577a826750941ef3ea9
I20260808 16:56:34.227669 9123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.218726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:42263 (local address 127.8.14.126:43709)
0808 16:56:34.218776 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:34.218786 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:34.219080 (+ 294us) server_negotiation.cc:407] Connection header received
0808 16:56:34.219999 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:34.220001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:34.220011 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:34.220066 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:34.220683 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.221203 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.222157 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.222351 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.222402 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.224479 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:34.224480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:34.224481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:34.224489 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:34.226066 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:34.226565 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:34.226565 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:34.226566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:34.226618 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:34.226890 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:34.226891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:34.226891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:34.227040 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:34.227095 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:34.227424 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 16:56:34.227542 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:56:34.227850 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.218717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:43709 (local address 127.8.14.65:42263)
0808 16:56:34.218935 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:34.218982 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:34.219909 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:34.220167 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:34.220170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:34.220407 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:34.220591 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:34.220595 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.221282 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.221283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:34.222079 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:34.222080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.222424 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.222425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:34.222521 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.223018 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:34.223026 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:34.224356 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:34.226146 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:34.226154 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:34.226156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:34.226488 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:34.226682 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:34.226683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:34.226684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:34.226827 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:34.227110 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:34.227116 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:34.227287 (+ 171us) client_negotiation.cc:769] Sending connection context
0808 16:56:34.227403 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 16:56:34.227593 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260808 16:56:34.228328 9346 heartbeater.cc:347] Connected to a master server at 127.8.14.126:43709
I20260808 16:56:34.228420 9346 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:34.228617 9346 heartbeater.cc:511] Master 127.8.14.126:43709 requested a full tablet report, sending...
I20260808 16:56:34.229153 9061 ts_manager.cc:194] Registered new tserver with Master: 0b6927589e7043d382491248f3449d88 (127.8.14.65:46067)
I20260808 16:56:34.229774 9061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:42263
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 16:56:34.249640 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:43709
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:41011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:34.334771 9354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:34.335079 9354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:34.335111 9354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:34.336275 9354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:34.336315 9354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:34.336344 9354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:56:34.337836 9354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:43709
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:34.338047 9354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:34.338239 9354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:34.343607 9361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:34.343709 9354 server_base.cc:1034] running on GCE node
W20260808 16:56:34.343626 9362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:34.343575 9364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:34.344075 9354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:34.344304 9354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:34.345440 9354 hybrid_clock.cc:648] HybridClock initialized: now 1786208194345427 us; error 32 us; skew 500 ppm
Aug 08 16:56:34 dist-test-slave-1nl6 krb5kdc[8997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208194, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:34.348116 9354 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:56:34.348636 9354 webserver.cc:460] Webserver started at http://127.8.14.66:42111/ using document root <none> and password file <none>
I20260808 16:56:34.348903 9354 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:34.348975 9354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:34.349128 9354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:34.350200 9354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "fca75eb153d64d4e9d6129e0e016a449"
format_stamp: "Formatted at 2026-08-08 16:56:34 on dist-test-slave-1nl6"
server_key: "1e2eb636a3fd7d22fa1eb047581bf229"
server_key_iv: "2492c3a6dfbcab1d188c0133b7a59d68"
server_key_version: "encryptionkey@0"
I20260808 16:56:34.350587 9354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "fca75eb153d64d4e9d6129e0e016a449"
format_stamp: "Formatted at 2026-08-08 16:56:34 on dist-test-slave-1nl6"
server_key: "1e2eb636a3fd7d22fa1eb047581bf229"
server_key_iv: "2492c3a6dfbcab1d188c0133b7a59d68"
server_key_version: "encryptionkey@0"
I20260808 16:56:34.352069 9354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:34.352883 9373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:34.353060 9354 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:34.353165 9354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "fca75eb153d64d4e9d6129e0e016a449"
format_stamp: "Formatted at 2026-08-08 16:56:34 on dist-test-slave-1nl6"
server_key: "1e2eb636a3fd7d22fa1eb047581bf229"
server_key_iv: "2492c3a6dfbcab1d188c0133b7a59d68"
server_key_version: "encryptionkey@0"
I20260808 16:56:34.353250 9354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:34.363297 9354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:34.365669 9354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:34.365850 9354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:34.367367 9354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:34.367738 9354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:34.367797 9354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:34.367843 9354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:34.367870 9354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:34.374061 9354 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:46799
I20260808 16:56:34.374104 9528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:46799 every 8 connection(s)
I20260808 16:56:34.374456 9354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
Aug 08 16:56:34 dist-test-slave-1nl6 krb5kdc[8997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208194, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:34.383514 9123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.375275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:47077 (local address 127.8.14.126:43709)
0808 16:56:34.375357 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:34.375359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:34.375550 (+ 191us) server_negotiation.cc:407] Connection header received
0808 16:56:34.376487 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:34.376488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:34.376498 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:34.376562 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:34.377167 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.377668 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.378634 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.378809 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.378858 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.381105 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:34.381107 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:34.381108 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:34.381114 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:34.382343 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:34.382706 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:34.382707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:34.382707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:34.382759 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:34.382999 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:34.382999 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:34.382999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:34.383104 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:34.383153 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:34.383383 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 16:56:34.383448 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 16:56:34.383675 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.375230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:43709 (local address 127.8.14.66:47077)
0808 16:56:34.375391 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:34.375434 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:34.376389 (+ 955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:34.376618 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:34.376621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:34.376889 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:34.377092 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:34.377096 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.377750 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.377751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:34.378560 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:34.378562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.378881 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.378882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:34.379053 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.379622 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:34.379631 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:34.381019 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:34.382417 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:34.382424 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:34.382426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:34.382640 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:34.382818 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:34.382819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:34.382819 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:34.382949 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:34.383156 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:34.383160 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:34.383269 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 16:56:34.383356 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 16:56:34.383474 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260808 16:56:34.384195 9529 heartbeater.cc:347] Connected to a master server at 127.8.14.126:43709
I20260808 16:56:34.384312 9529 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:34.384510 9529 heartbeater.cc:511] Master 127.8.14.126:43709 requested a full tablet report, sending...
I20260808 16:56:34.384630 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 9354
I20260808 16:56:34.384733 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:56:34.384915 8249 external_mini_cluster.cc:1428] Setting key 34049c1c89d75708d0349a6d7231d803
I20260808 16:56:34.385001 9061 ts_manager.cc:194] Registered new tserver with Master: fca75eb153d64d4e9d6129e0e016a449 (127.8.14.66:46799)
I20260808 16:56:34.385702 9061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:47077
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 16:56:34.411845 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:43709
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:41011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:34.495990 9538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:34.496213 9538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:34.496236 9538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:34.497409 9538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:34.497452 9538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:34.497478 9538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:56:34.498940 9538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:43709
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:34.499142 9538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:34.499331 9538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:34.504585 9551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:34.504585 9549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:34.504616 9558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:34.504737 9538 server_base.cc:1034] running on GCE node
I20260808 16:56:34.505048 9538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:34.505278 9538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:34.506415 9538 hybrid_clock.cc:648] HybridClock initialized: now 1786208194506396 us; error 33 us; skew 500 ppm
Aug 08 16:56:34 dist-test-slave-1nl6 krb5kdc[8997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208194, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:34.509063 9538 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:56:34.509511 9538 webserver.cc:460] Webserver started at http://127.8.14.67:39419/ using document root <none> and password file <none>
I20260808 16:56:34.509745 9538 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:34.509797 9538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:34.509967 9538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:34.511035 9538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "0859b476509846d39786a7dcd35960ee"
format_stamp: "Formatted at 2026-08-08 16:56:34 on dist-test-slave-1nl6"
server_key: "bfd32cd3225dc3b58b4f588332f4a94b"
server_key_iv: "b47018313c62ff9e6129774a6a8ff53a"
server_key_version: "encryptionkey@0"
I20260808 16:56:34.511406 9538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "0859b476509846d39786a7dcd35960ee"
format_stamp: "Formatted at 2026-08-08 16:56:34 on dist-test-slave-1nl6"
server_key: "bfd32cd3225dc3b58b4f588332f4a94b"
server_key_iv: "b47018313c62ff9e6129774a6a8ff53a"
server_key_version: "encryptionkey@0"
I20260808 16:56:34.512715 9538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:56:34.513406 9568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:34.513574 9538 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:34.513674 9538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "0859b476509846d39786a7dcd35960ee"
format_stamp: "Formatted at 2026-08-08 16:56:34 on dist-test-slave-1nl6"
server_key: "bfd32cd3225dc3b58b4f588332f4a94b"
server_key_iv: "b47018313c62ff9e6129774a6a8ff53a"
server_key_version: "encryptionkey@0"
I20260808 16:56:34.513751 9538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:34.522691 9538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:34.524499 9538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:34.524663 9538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:34.524892 9538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:34.525195 9538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:34.525254 9538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:34.525290 9538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:34.525321 9538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:34.531235 9538 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:45629
I20260808 16:56:34.531288 9733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:45629 every 8 connection(s)
I20260808 16:56:34.531586 9538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
Aug 08 16:56:34 dist-test-slave-1nl6 krb5kdc[8997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208194, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:34.537907 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 9538
I20260808 16:56:34.538002 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:56:34.538169 8249 external_mini_cluster.cc:1428] Setting key 95f906f90877e99fa16572a918de8361
I20260808 16:56:34.540856 9123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.532268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:40237 (local address 127.8.14.126:43709)
0808 16:56:34.532314 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:34.532316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:34.532511 (+ 195us) server_negotiation.cc:407] Connection header received
0808 16:56:34.533485 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:34.533487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:34.533500 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:34.533551 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:34.534159 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.534660 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.535722 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.535944 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.536033 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.538245 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:34.538247 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:34.538248 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:34.538256 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:34.539735 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:34.540129 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:34.540130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:34.540130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:34.540180 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:34.540370 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:34.540371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:34.540371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:34.540469 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:34.540519 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:34.540736 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 16:56:34.540795 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:56:34.541038 9739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.532237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:43709 (local address 127.8.14.67:40237)
0808 16:56:34.532395 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:34.532429 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:34.533357 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:34.533623 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:34.533627 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:34.533879 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:34.534057 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:34.534061 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.534759 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.534760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:34.535624 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:34.535627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.536043 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.536045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:34.536173 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.536686 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:34.536694 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:34.538120 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:34.539818 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:34.539823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:34.539825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:34.540061 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:34.540221 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:34.540221 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:34.540221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:34.540311 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:34.540527 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:34.540532 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:34.540639 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 16:56:34.540720 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 16:56:34.540844 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260808 16:56:34.541471 9736 heartbeater.cc:347] Connected to a master server at 127.8.14.126:43709
I20260808 16:56:34.541548 9736 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:34.541682 9736 heartbeater.cc:511] Master 127.8.14.126:43709 requested a full tablet report, sending...
I20260808 16:56:34.541960 9061 ts_manager.cc:194] Registered new tserver with Master: 0859b476509846d39786a7dcd35960ee (127.8.14.67:45629)
I20260808 16:56:34.542346 9061 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:40237
I20260808 16:56:34.549986 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:56:34.557057 9123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.551463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40880 (local address 127.8.14.126:43709)
0808 16:56:34.551515 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:34.551517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:34.551589 (+ 72us) server_negotiation.cc:407] Connection header received
0808 16:56:34.551635 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:34.551636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:34.551644 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:34.551688 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:34.552237 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.552719 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.553390 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.553557 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.553604 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.554719 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:34.554719 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:34.554720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:34.554724 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:34.555969 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:34.556341 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:34.556342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:34.556342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:34.556393 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:34.556552 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:34.556553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:34.556553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:34.556725 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:34.556810 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:34.556902 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 16:56:34.556962 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 16:56:34.559139 9061 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:56:34.560681 9061 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:56:34.567173 9749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.563218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:46067 (local address 127.0.0.1:53400)
0808 16:56:34.563581 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:34.563591 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:34.563632 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:34.563887 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:34.563888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:34.563890 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:34.563943 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:34.563944 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.564829 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.564830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:34.565938 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:34.565940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.566844 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.566845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:34.566952 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.566966 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 16:56:34.567038 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 16:56:34.567089 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":35,"thread_start_us":81,"threads_started":1}
I20260808 16:56:34.567216 9752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.563341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53400 (local address 127.8.14.65:46067)
0808 16:56:34.563718 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:34.563722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:34.563731 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:56:34.563769 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:34.563771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:34.563812 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:34.563889 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:34.564001 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.564729 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.566031 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.566753 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.566827 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.566883 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:34.567074 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 16:56:34.567133 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":84,"threads_started":1}
I20260808 16:56:34.567440 9748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.562995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:46799 (local address 127.0.0.1:45198)
0808 16:56:34.563111 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:34.563152 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:34.563222 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:34.563468 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:34.563471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:34.563474 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:34.563541 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:34.563542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.564558 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.564559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:34.565926 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:34.565930 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.566938 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.566940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:34.567042 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.567047 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:56:34.567103 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 16:56:34.567156 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":1,"thread_start_us":63,"threads_started":1}
I20260808 16:56:34.567571 9750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.563095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45198 (local address 127.8.14.66:46799)
0808 16:56:34.563301 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:34.563305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:34.563314 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:56:34.563351 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:34.563352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:34.563367 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:34.563461 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:34.563626 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.564457 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.566031 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.566760 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.566826 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.566883 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:34.567214 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 16:56:34.567256 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":86,"threads_started":1}
I20260808 16:56:34.567870 9250 tablet_service.cc:1467] Processing CreateTablet for tablet 794898c4e0c049bd88695cd5808b1953 (DEFAULT_TABLE table=test-table [id=06d8b71a14c64cd4bfb513f60942b9bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:34.568007 9751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.562995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:45629 (local address 127.0.0.1:58044)
0808 16:56:34.563517 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:34.563528 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:34.563601 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:34.564740 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:34.564741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:34.564744 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:34.564821 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:34.564823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.565667 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.565668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:34.566541 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:34.566544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.567762 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.567763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:34.567836 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.567839 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:56:34.567909 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 16:56:34.567945 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":26,"thread_start_us":79,"threads_started":1}
I20260808 16:56:34.568025 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.563364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58044 (local address 127.8.14.67:45629)
0808 16:56:34.563535 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:34.563541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:34.564580 (+ 1039us) server_negotiation.cc:407] Connection header received
0808 16:56:34.564624 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:34.564626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:34.564641 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:34.564717 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:34.564914 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.565562 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.566640 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.567613 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.567679 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.567729 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:34.567923 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 16:56:34.567957 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":75,"threads_started":1}
I20260808 16:56:34.568270 9250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 794898c4e0c049bd88695cd5808b1953. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:34.568727 9632 tablet_service.cc:1467] Processing CreateTablet for tablet 794898c4e0c049bd88695cd5808b1953 (DEFAULT_TABLE table=test-table [id=06d8b71a14c64cd4bfb513f60942b9bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:34.568971 9632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 794898c4e0c049bd88695cd5808b1953. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:34.571213 9758 tablet_bootstrap.cc:492] T 794898c4e0c049bd88695cd5808b1953 P 0b6927589e7043d382491248f3449d88: Bootstrap starting.
I20260808 16:56:34.571899 9758 tablet_bootstrap.cc:654] T 794898c4e0c049bd88695cd5808b1953 P 0b6927589e7043d382491248f3449d88: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:34.571832 9441 tablet_service.cc:1467] Processing CreateTablet for tablet 794898c4e0c049bd88695cd5808b1953 (DEFAULT_TABLE table=test-table [id=06d8b71a14c64cd4bfb513f60942b9bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:34.572185 9441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 794898c4e0c049bd88695cd5808b1953. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:34.572525 9758 log.cc:824] T 794898c4e0c049bd88695cd5808b1953 P 0b6927589e7043d382491248f3449d88: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:34.573321 9758 tablet_bootstrap.cc:492] T 794898c4e0c049bd88695cd5808b1953 P 0b6927589e7043d382491248f3449d88: No bootstrap required, opened a new log
I20260808 16:56:34.573398 9758 ts_tablet_manager.cc:1397] T 794898c4e0c049bd88695cd5808b1953 P 0b6927589e7043d382491248f3449d88: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:56:34.575477 9763 tablet_bootstrap.cc:492] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449: Bootstrap starting.
I20260808 16:56:34.575533 9758 raft_consensus.cc:348] T 794898c4e0c049bd88695cd5808b1953 P 0b6927589e7043d382491248f3449d88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0859b476509846d39786a7dcd35960ee" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45629 } } peers { permanent_uuid: "0b6927589e7043d382491248f3449d88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 46067 } } peers { permanent_uuid: "fca75eb153d64d4e9d6129e0e016a449" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46799 } }
I20260808 16:56:34.575729 9758 raft_consensus.cc:374] T 794898c4e0c049bd88695cd5808b1953 P 0b6927589e7043d382491248f3449d88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:34.575789 9758 raft_consensus.cc:729] T 794898c4e0c049bd88695cd5808b1953 P 0b6927589e7043d382491248f3449d88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6927589e7043d382491248f3449d88, State: Initialized, Role: FOLLOWER
I20260808 16:56:34.575852 9760 tablet_bootstrap.cc:492] T 794898c4e0c049bd88695cd5808b1953 P 0859b476509846d39786a7dcd35960ee: Bootstrap starting.
I20260808 16:56:34.576115 9763 tablet_bootstrap.cc:654] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:34.576049 9758 consensus_queue.cc:260] T 794898c4e0c049bd88695cd5808b1953 P 0b6927589e7043d382491248f3449d88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0859b476509846d39786a7dcd35960ee" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45629 } } peers { permanent_uuid: "0b6927589e7043d382491248f3449d88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 46067 } } peers { permanent_uuid: "fca75eb153d64d4e9d6129e0e016a449" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46799 } }
I20260808 16:56:34.576400 9758 ts_tablet_manager.cc:1428] T 794898c4e0c049bd88695cd5808b1953 P 0b6927589e7043d382491248f3449d88: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:56:34.576501 9760 tablet_bootstrap.cc:654] T 794898c4e0c049bd88695cd5808b1953 P 0859b476509846d39786a7dcd35960ee: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:34.576534 9346 heartbeater.cc:503] Master 127.8.14.126:43709 was elected leader, sending a full tablet report...
I20260808 16:56:34.576748 9763 log.cc:824] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:34.577068 9760 log.cc:824] T 794898c4e0c049bd88695cd5808b1953 P 0859b476509846d39786a7dcd35960ee: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:34.577497 9763 tablet_bootstrap.cc:492] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449: No bootstrap required, opened a new log
I20260808 16:56:34.577587 9763 ts_tablet_manager.cc:1397] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:56:34.577880 9760 tablet_bootstrap.cc:492] T 794898c4e0c049bd88695cd5808b1953 P 0859b476509846d39786a7dcd35960ee: No bootstrap required, opened a new log
I20260808 16:56:34.577948 9760 ts_tablet_manager.cc:1397] T 794898c4e0c049bd88695cd5808b1953 P 0859b476509846d39786a7dcd35960ee: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:56:34.579686 9763 raft_consensus.cc:348] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0859b476509846d39786a7dcd35960ee" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45629 } } peers { permanent_uuid: "0b6927589e7043d382491248f3449d88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 46067 } } peers { permanent_uuid: "fca75eb153d64d4e9d6129e0e016a449" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46799 } }
I20260808 16:56:34.579865 9763 raft_consensus.cc:374] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:34.579906 9763 raft_consensus.cc:729] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fca75eb153d64d4e9d6129e0e016a449, State: Initialized, Role: FOLLOWER
I20260808 16:56:34.579924 9760 raft_consensus.cc:348] T 794898c4e0c049bd88695cd5808b1953 P 0859b476509846d39786a7dcd35960ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0859b476509846d39786a7dcd35960ee" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45629 } } peers { permanent_uuid: "0b6927589e7043d382491248f3449d88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 46067 } } peers { permanent_uuid: "fca75eb153d64d4e9d6129e0e016a449" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46799 } }
I20260808 16:56:34.580022 9763 consensus_queue.cc:260] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0859b476509846d39786a7dcd35960ee" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45629 } } peers { permanent_uuid: "0b6927589e7043d382491248f3449d88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 46067 } } peers { permanent_uuid: "fca75eb153d64d4e9d6129e0e016a449" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46799 } }
I20260808 16:56:34.580078 9760 raft_consensus.cc:374] T 794898c4e0c049bd88695cd5808b1953 P 0859b476509846d39786a7dcd35960ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:34.580114 9760 raft_consensus.cc:729] T 794898c4e0c049bd88695cd5808b1953 P 0859b476509846d39786a7dcd35960ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0859b476509846d39786a7dcd35960ee, State: Initialized, Role: FOLLOWER
I20260808 16:56:34.580230 9760 consensus_queue.cc:260] T 794898c4e0c049bd88695cd5808b1953 P 0859b476509846d39786a7dcd35960ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0859b476509846d39786a7dcd35960ee" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45629 } } peers { permanent_uuid: "0b6927589e7043d382491248f3449d88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 46067 } } peers { permanent_uuid: "fca75eb153d64d4e9d6129e0e016a449" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46799 } }
I20260808 16:56:34.580295 9529 heartbeater.cc:503] Master 127.8.14.126:43709 was elected leader, sending a full tablet report...
I20260808 16:56:34.580317 9763 ts_tablet_manager.cc:1428] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:56:34.580492 9760 ts_tablet_manager.cc:1428] T 794898c4e0c049bd88695cd5808b1953 P 0859b476509846d39786a7dcd35960ee: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 16:56:34.580552 9736 heartbeater.cc:503] Master 127.8.14.126:43709 was elected leader, sending a full tablet report...
W20260808 16:56:34.625006 9531 tablet.cc:2369] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:56:34.718828 9347 tablet.cc:2369] T 794898c4e0c049bd88695cd5808b1953 P 0b6927589e7043d382491248f3449d88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:56:34.780648 9769 raft_consensus.cc:482] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:56:34.780778 9769 raft_consensus.cc:504] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0859b476509846d39786a7dcd35960ee" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45629 } } peers { permanent_uuid: "0b6927589e7043d382491248f3449d88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 46067 } } peers { permanent_uuid: "fca75eb153d64d4e9d6129e0e016a449" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46799 } }
I20260808 16:56:34.781139 9769 leader_election.cc:288] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0859b476509846d39786a7dcd35960ee (127.8.14.67:45629), 0b6927589e7043d382491248f3449d88 (127.8.14.65:46067)
W20260808 16:56:34.782130 9738 tablet.cc:2369] T 794898c4e0c049bd88695cd5808b1953 P 0859b476509846d39786a7dcd35960ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:56:34.784279 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.781453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:45823 (local address 127.8.14.67:45629)
0808 16:56:34.781517 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:34.781519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:34.781529 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:56:34.781633 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:34.781633 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:34.781643 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:34.781714 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:34.781867 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.782527 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.783530 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.783924 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.784007 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.784018 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:34.784176 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 16:56:34.784212 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 16:56:34.784315 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.781345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:45629 (local address 127.8.14.66:45823)
0808 16:56:34.781420 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:34.781438 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:34.781512 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:34.781719 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:34.781720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:34.781722 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:34.781797 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:34.781799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.782617 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.782619 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:34.783439 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:34.783442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.784019 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.784020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:34.784093 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.784096 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:56:34.784155 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 16:56:34.784198 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 16:56:34.784320 9752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.781544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:55115 (local address 127.8.14.65:46067)
0808 16:56:34.781596 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:34.781598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:34.781725 (+ 127us) server_negotiation.cc:407] Connection header received
0808 16:56:34.781820 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:34.781821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:34.781832 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:34.781900 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:34.782060 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.782609 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.783600 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.783961 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.784042 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.784052 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:34.784207 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 16:56:34.784251 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 16:56:34.784338 9772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.781468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:46067 (local address 127.8.14.66:55115)
0808 16:56:34.781648 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:34.781657 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:34.781739 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:34.781903 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:34.781903 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:34.781905 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:34.781978 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:34.781980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.782699 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.782700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:34.783507 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:34.783509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.784043 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.784044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:34.784117 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.784121 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:56:34.784198 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 16:56:34.784238 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":128,"threads_started":1}
I20260808 16:56:34.784673 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794898c4e0c049bd88695cd5808b1953" candidate_uuid: "fca75eb153d64d4e9d6129e0e016a449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6927589e7043d382491248f3449d88" is_pre_election: true
I20260808 16:56:34.784847 9283 raft_consensus.cc:2454] T 794898c4e0c049bd88695cd5808b1953 P 0b6927589e7043d382491248f3449d88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fca75eb153d64d4e9d6129e0e016a449 in term 0.
I20260808 16:56:34.784972 9662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794898c4e0c049bd88695cd5808b1953" candidate_uuid: "fca75eb153d64d4e9d6129e0e016a449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0859b476509846d39786a7dcd35960ee" is_pre_election: true
I20260808 16:56:34.785108 9662 raft_consensus.cc:2454] T 794898c4e0c049bd88695cd5808b1953 P 0859b476509846d39786a7dcd35960ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fca75eb153d64d4e9d6129e0e016a449 in term 0.
I20260808 16:56:34.785100 9382 leader_election.cc:302] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6927589e7043d382491248f3449d88, fca75eb153d64d4e9d6129e0e016a449; no voters:
I20260808 16:56:34.785238 9769 raft_consensus.cc:2781] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:56:34.785311 9769 raft_consensus.cc:482] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:56:34.785346 9769 raft_consensus.cc:3037] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:34.786024 9769 raft_consensus.cc:504] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0859b476509846d39786a7dcd35960ee" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45629 } } peers { permanent_uuid: "0b6927589e7043d382491248f3449d88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 46067 } } peers { permanent_uuid: "fca75eb153d64d4e9d6129e0e016a449" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46799 } }
I20260808 16:56:34.786161 9769 leader_election.cc:288] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 [CANDIDATE]: Term 1 election: Requested vote from peers 0859b476509846d39786a7dcd35960ee (127.8.14.67:45629), 0b6927589e7043d382491248f3449d88 (127.8.14.65:46067)
I20260808 16:56:34.786306 9662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794898c4e0c049bd88695cd5808b1953" candidate_uuid: "fca75eb153d64d4e9d6129e0e016a449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0859b476509846d39786a7dcd35960ee"
I20260808 16:56:34.786394 9662 raft_consensus.cc:3037] T 794898c4e0c049bd88695cd5808b1953 P 0859b476509846d39786a7dcd35960ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:34.786422 9283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794898c4e0c049bd88695cd5808b1953" candidate_uuid: "fca75eb153d64d4e9d6129e0e016a449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6927589e7043d382491248f3449d88"
I20260808 16:56:34.786497 9283 raft_consensus.cc:3037] T 794898c4e0c049bd88695cd5808b1953 P 0b6927589e7043d382491248f3449d88 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:34.787247 9662 raft_consensus.cc:2454] T 794898c4e0c049bd88695cd5808b1953 P 0859b476509846d39786a7dcd35960ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fca75eb153d64d4e9d6129e0e016a449 in term 1.
I20260808 16:56:34.787318 9283 raft_consensus.cc:2454] T 794898c4e0c049bd88695cd5808b1953 P 0b6927589e7043d382491248f3449d88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fca75eb153d64d4e9d6129e0e016a449 in term 1.
I20260808 16:56:34.787408 9382 leader_election.cc:302] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0859b476509846d39786a7dcd35960ee, fca75eb153d64d4e9d6129e0e016a449; no voters:
I20260808 16:56:34.787503 9769 raft_consensus.cc:2781] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:34.787667 9769 raft_consensus.cc:686] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 [term 1 LEADER]: Becoming Leader. State: Replica: fca75eb153d64d4e9d6129e0e016a449, State: Running, Role: LEADER
I20260808 16:56:34.787768 9769 consensus_queue.cc:237] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0859b476509846d39786a7dcd35960ee" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45629 } } peers { permanent_uuid: "0b6927589e7043d382491248f3449d88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 46067 } } peers { permanent_uuid: "fca75eb153d64d4e9d6129e0e016a449" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46799 } }
I20260808 16:56:34.788609 9060 catalog_manager.cc:5495] T 794898c4e0c049bd88695cd5808b1953 P fca75eb153d64d4e9d6129e0e016a449 reported cstate change: term changed from 0 to 1, leader changed from <none> to fca75eb153d64d4e9d6129e0e016a449 (127.8.14.66). New cstate: current_term: 1 leader_uuid: "fca75eb153d64d4e9d6129e0e016a449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0859b476509846d39786a7dcd35960ee" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b6927589e7043d382491248f3449d88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 46067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fca75eb153d64d4e9d6129e0e016a449" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46799 } health_report { overall_health: HEALTHY } } }
Aug 08 16:56:34 dist-test-slave-1nl6 krb5kdc[8997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.65@KRBTEST.COM
I20260808 16:56:34.852664 9752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.845599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53412 (local address 127.8.14.65:46067)
0808 16:56:34.845684 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:34.845686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:34.845737 (+ 51us) server_negotiation.cc:407] Connection header received
0808 16:56:34.845797 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:34.845798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:34.845808 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:34.845852 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:34.846392 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.846866 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.847591 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.847769 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.847819 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.850083 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:34.850094 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:34.850095 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:34.850101 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:34.851692 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:34.852029 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:34.852032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:34.852032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:34.852095 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:34.852263 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:34.852263 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:34.852263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:34.852384 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:34.852490 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:34.852538 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 16:56:34.852599 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 16:56:34 dist-test-slave-1nl6 krb5kdc[8997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208194, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 16:56:34 dist-test-slave-1nl6 krb5kdc[8997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208194, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.14.65@KRBTEST.COM
I20260808 16:56:34.877288 9752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.870612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53416 (local address 127.8.14.65:46067)
0808 16:56:34.870681 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:34.870683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:34.870739 (+ 56us) server_negotiation.cc:407] Connection header received
0808 16:56:34.870786 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:34.870787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:34.870796 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:34.870851 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:34.871454 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.871899 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.872616 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.872790 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.872839 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.874941 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:34.874943 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:34.874943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:34.874952 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:34.876353 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:34.876650 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:34.876651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:34.876651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:34.876701 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:34.876867 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:34.876867 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:34.876867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:34.876950 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:34.877019 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:34.877135 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 16:56:34.877190 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 16:56:34 dist-test-slave-1nl6 krb5kdc[8997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 16:56:34 dist-test-slave-1nl6 krb5kdc[8997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.65@KRBTEST.COM
I20260808 16:56:34.899377 9752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:34.893017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53432 (local address 127.8.14.65:46067)
0808 16:56:34.893097 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:34.893099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:34.893144 (+ 45us) server_negotiation.cc:407] Connection header received
0808 16:56:34.893197 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:34.893198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:34.893207 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:34.893255 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:34.893744 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.894187 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.894859 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:34.895077 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:34.895129 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:34.897172 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:34.897173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:34.897173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:34.897180 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:34.898336 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:34.898632 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:34.898633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:34.898634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:34.898682 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:34.898851 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:34.898852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:34.898852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:34.899065 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:34.899162 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:34.899235 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 16:56:34.899292 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 16:56:34.899909 9218 tablet_service.cc:2908] Rejecting scan request for tablet 794898c4e0c049bd88695cd5808b1953: Uninitialized: clean time has not yet been initialized
I20260808 16:56:34.900872 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 9162
I20260808 16:56:34.913640 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 9354
I20260808 16:56:34.925011 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 9538
I20260808 16:56:34.931777 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 9016
2026-08-08T16:56:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2404 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:34 dist-test-slave-1nl6 krb5kdc[9793](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:34 dist-test-slave-1nl6 krb5kdc[9793](info): set up 2 sockets
Aug 08 16:56:34 dist-test-slave-1nl6 krb5kdc[9793](info): commencing operation
krb5kdc: starting...
W20260808 16:56:36.155810 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:56:36 dist-test-slave-1nl6 krb5kdc[9793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:56:36.275306 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:40937
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:39639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:40937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:36.358381 9808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:36.358578 9808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:36.358599 9808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:36.359786 9808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:36.359828 9808 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 16:56:36.359841 9808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:36.359854 9808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:36.359867 9808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:36.359925 9808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:36.361353 9808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:40937
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:40937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:36.361619 9808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:36.361867 9808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:36.366763 9816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:36.366953 9808 server_base.cc:1034] running on GCE node
W20260808 16:56:36.366758 9815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:36.366752 9820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:36.367298 9808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:36.367606 9808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:36.368774 9808 hybrid_clock.cc:648] HybridClock initialized: now 1786208196368757 us; error 39 us; skew 500 ppm
Aug 08 16:56:36 dist-test-slave-1nl6 krb5kdc[9793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208196, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:36.371477 9808 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:56:36.371833 9808 webserver.cc:460] Webserver started at http://127.8.14.126:42615/ using document root <none> and password file <none>
I20260808 16:56:36.372025 9808 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:36.372066 9808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:36.372185 9808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:36.373152 9808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "d8d4f59aaf93434e93ab21373ff99c2b"
format_stamp: "Formatted at 2026-08-08 16:56:36 on dist-test-slave-1nl6"
server_key: "a940b25ca7502351b3f273ebc99dd4e3"
server_key_iv: "303ce4e3bbfd230a5c53adc4f9ebd5d3"
server_key_version: "encryptionkey@0"
I20260808 16:56:36.373458 9808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "d8d4f59aaf93434e93ab21373ff99c2b"
format_stamp: "Formatted at 2026-08-08 16:56:36 on dist-test-slave-1nl6"
server_key: "a940b25ca7502351b3f273ebc99dd4e3"
server_key_iv: "303ce4e3bbfd230a5c53adc4f9ebd5d3"
server_key_version: "encryptionkey@0"
I20260808 16:56:36.374719 9808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 16:56:36.375485 9833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:36.375665 9808 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:36.375779 9808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "d8d4f59aaf93434e93ab21373ff99c2b"
format_stamp: "Formatted at 2026-08-08 16:56:36 on dist-test-slave-1nl6"
server_key: "a940b25ca7502351b3f273ebc99dd4e3"
server_key_iv: "303ce4e3bbfd230a5c53adc4f9ebd5d3"
server_key_version: "encryptionkey@0"
I20260808 16:56:36.375860 9808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:36.403013 9808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:36.404855 9808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:36.405032 9808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:36.409746 9808 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:40937
I20260808 16:56:36.409850 9918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:40937 every 8 connection(s)
I20260808 16:56:36.410181 9808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:36.410862 9919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:36.413169 9919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b: Bootstrap starting.
I20260808 16:56:36.413772 9919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:36.414088 9919 log.cc:824] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:36.414799 9919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b: No bootstrap required, opened a new log
I20260808 16:56:36.416980 9919 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d4f59aaf93434e93ab21373ff99c2b" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40937 } }
I20260808 16:56:36.417119 9919 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:36.417142 9919 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8d4f59aaf93434e93ab21373ff99c2b, State: Initialized, Role: FOLLOWER
I20260808 16:56:36.417323 9919 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d4f59aaf93434e93ab21373ff99c2b" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40937 } }
I20260808 16:56:36.417423 9919 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:36.417480 9919 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:36.417534 9919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:36.418100 9919 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d4f59aaf93434e93ab21373ff99c2b" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40937 } }
I20260808 16:56:36.418219 9919 leader_election.cc:302] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8d4f59aaf93434e93ab21373ff99c2b; no voters:
I20260808 16:56:36.418397 9919 leader_election.cc:288] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:36.418431 9924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:36.418638 9924 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [term 1 LEADER]: Becoming Leader. State: Replica: d8d4f59aaf93434e93ab21373ff99c2b, State: Running, Role: LEADER
I20260808 16:56:36.418668 9919 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:36.418799 9924 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d4f59aaf93434e93ab21373ff99c2b" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40937 } }
I20260808 16:56:36.419205 9926 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8d4f59aaf93434e93ab21373ff99c2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d4f59aaf93434e93ab21373ff99c2b" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40937 } } }
I20260808 16:56:36.419209 9927 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8d4f59aaf93434e93ab21373ff99c2b. Latest consensus state: current_term: 1 leader_uuid: "d8d4f59aaf93434e93ab21373ff99c2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d4f59aaf93434e93ab21373ff99c2b" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40937 } } }
I20260808 16:56:36.419356 9926 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:36.419389 9927 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:36.419574 9934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:36.420081 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 9808
I20260808 16:56:36.420181 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:56:36.420234 9934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:36.420375 8249 external_mini_cluster.cc:1428] Setting key 836a98768d7a097b99d859c1e3b7fec9
I20260808 16:56:36.422113 9934 catalog_manager.cc:1347] Generated new cluster ID: 94256bf9f2994bdbae2890e515f762a8
I20260808 16:56:36.422168 9934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:56:36 dist-test-slave-1nl6 krb5kdc[9793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:36.431433 9934 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:36.432138 9934 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:36.432241 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.420818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39004 (local address 127.8.14.126:40937)
0808 16:56:36.421198 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:36.421206 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:36.421231 (+ 25us) server_negotiation.cc:407] Connection header received
0808 16:56:36.421328 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:36.421333 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:36.421373 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:36.422362 (+ 989us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:36.422452 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.423871 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.424900 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.425496 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.425854 (+ 358us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.428219 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:36.428223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:36.428224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:36.428235 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:36.430161 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:36.430562 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:36.430565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:36.430565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:36.430659 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:36.430863 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:36.430863 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:36.430863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:36.431171 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:36.431771 (+ 600us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:36.431841 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 16:56:36.431972 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":85,"threads_started":1}
I20260808 16:56:36.436774 9934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b: Generated new TSK 0
I20260808 16:56:36.436971 9934 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 16:56:36.463935 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:40937
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:39639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:36.546693 9953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:36.546960 9953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:36.547001 9953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:36.548113 9953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:36.548149 9953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:36.548169 9953 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:56:36.548187 9953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:56:36.549621 9953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.14.126:40937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:36.549803 9953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:36.549973 9953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:36.554813 9960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:36.554857 9961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:36.554996 9953 server_base.cc:1034] running on GCE node
W20260808 16:56:36.554883 9963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:36.555261 9953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:36.555517 9953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:36.556666 9953 hybrid_clock.cc:648] HybridClock initialized: now 1786208196556640 us; error 42 us; skew 500 ppm
Aug 08 16:56:36 dist-test-slave-1nl6 krb5kdc[9793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208196, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:36.559398 9953 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:56:36.559892 9953 webserver.cc:460] Webserver started at http://127.8.14.65:36507/ using document root <none> and password file <none>
I20260808 16:56:36.560114 9953 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:36.560158 9953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:36.560252 9953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:36.561305 9953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "d3985510e5db45529e0de98fa65a1853"
format_stamp: "Formatted at 2026-08-08 16:56:36 on dist-test-slave-1nl6"
server_key: "1ec59a4d5faeeee439336d274045a2eb"
server_key_iv: "df289081d483361b26733e463ca2a972"
server_key_version: "encryptionkey@0"
I20260808 16:56:36.561622 9953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "d3985510e5db45529e0de98fa65a1853"
format_stamp: "Formatted at 2026-08-08 16:56:36 on dist-test-slave-1nl6"
server_key: "1ec59a4d5faeeee439336d274045a2eb"
server_key_iv: "df289081d483361b26733e463ca2a972"
server_key_version: "encryptionkey@0"
I20260808 16:56:36.562973 9953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 16:56:36.563805 9973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:36.563987 9953 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:56:36.564102 9953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "d3985510e5db45529e0de98fa65a1853"
format_stamp: "Formatted at 2026-08-08 16:56:36 on dist-test-slave-1nl6"
server_key: "1ec59a4d5faeeee439336d274045a2eb"
server_key_iv: "df289081d483361b26733e463ca2a972"
server_key_version: "encryptionkey@0"
I20260808 16:56:36.564184 9953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:36.578605 9953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:36.580578 9953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:36.580744 9953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:36.581255 9953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:36.581324 9953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:36.581370 9953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:36.581403 9953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 16:56:36 dist-test-slave-1nl6 krb5kdc[9793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208196, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:36.588956 9953 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:40855
I20260808 16:56:36.589020 10138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:40855 every 8 connection(s)
I20260808 16:56:36.589414 9953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:56:36.591298 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.581900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:60055 (local address 127.8.14.126:40937)
0808 16:56:36.581943 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:36.581946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:36.582201 (+ 255us) server_negotiation.cc:407] Connection header received
0808 16:56:36.582814 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:36.582816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:36.582835 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:36.582932 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:36.583519 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.584396 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.585434 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.585657 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.585748 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.588370 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:36.588372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:36.588372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:36.588380 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:36.590116 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:36.590481 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:36.590482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:36.590482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:36.590545 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:36.590733 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:36.590733 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:36.590734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:36.590837 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:36.590885 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:36.591164 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 16:56:36.591235 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:56:36.591445 9983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.581903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40937 (local address 127.8.14.65:60055)
0808 16:56:36.582081 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:36.582116 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:36.582653 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:36.583032 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:36.583036 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:36.583261 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:36.583440 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.583444 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.584521 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.584522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.585332 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.585334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.585773 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.585775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.585894 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.586432 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:36.586443 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:36.588263 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:36.590193 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:36.590197 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:36.590199 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:36.590415 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:36.590600 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:36.590600 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:36.590600 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:36.590685 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:36.590876 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:36.590878 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:36.591055 (+ 177us) client_negotiation.cc:769] Sending connection context
0808 16:56:36.591137 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 16:56:36.591251 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":100,"threads_started":1}
I20260808 16:56:36.591951 10140 heartbeater.cc:347] Connected to a master server at 127.8.14.126:40937
I20260808 16:56:36.592038 10140 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:36.592240 10140 heartbeater.cc:511] Master 127.8.14.126:40937 requested a full tablet report, sending...
I20260808 16:56:36.592734 9855 ts_manager.cc:194] Registered new tserver with Master: d3985510e5db45529e0de98fa65a1853 (127.8.14.65:40855)
I20260808 16:56:36.593586 9855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:60055
I20260808 16:56:36.597968 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.593175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:39997 (local address 127.8.14.126:40937)
0808 16:56:36.593215 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:36.593217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:36.593263 (+ 46us) server_negotiation.cc:407] Connection header received
0808 16:56:36.593335 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:36.593336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:36.593343 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:36.593383 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:36.593776 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.594229 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.594946 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.595137 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.595188 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.596026 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:36.596027 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:36.596027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:36.596032 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:36.597121 (+ 1089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:36.597417 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:36.597417 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:36.597418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:36.597463 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:36.597627 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:36.597627 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:36.597627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:36.597710 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:36.597757 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:36.597864 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 16:56:36.597918 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:56:36.597977 10148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.593073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40937 (local address 127.8.14.65:39997)
0808 16:56:36.593206 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:36.593217 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:36.593275 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:36.593395 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:36.593395 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:36.593584 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:36.593656 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.593658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.594333 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.594334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.594830 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.594832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.595203 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.595204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.595262 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.595679 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:36.595685 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:36.595954 (+ 269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:36.597194 (+ 1240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:36.597195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:36.597195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:36.597353 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:36.597521 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:36.597522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:36.597522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:36.597574 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:36.597763 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:36.597763 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:36.597796 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 16:56:36.597849 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 16:56:36.597920 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
I20260808 16:56:36.599422 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 9953
I20260808 16:56:36.599534 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:56:36.599720 8249 external_mini_cluster.cc:1428] Setting key 34efb0677584c4ce1319470d6a6f88c1
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 16:56:36.623975 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:40937
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:39639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:36.705565 10151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:36.705758 10151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:36.705782 10151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:36.707017 10151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:36.707062 10151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:36.707082 10151 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:56:36.707140 10151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:56:36.708518 10151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.14.126:40937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:36.708766 10151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:36.709008 10151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:36.714047 10160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:36.714059 10162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:36.714190 10151 server_base.cc:1034] running on GCE node
W20260808 16:56:36.714059 10164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:36.714447 10151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:36.714701 10151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:36.715844 10151 hybrid_clock.cc:648] HybridClock initialized: now 1786208196715830 us; error 40 us; skew 500 ppm
Aug 08 16:56:36 dist-test-slave-1nl6 krb5kdc[9793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208196, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:36.718293 10151 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:56:36.718734 10151 webserver.cc:460] Webserver started at http://127.8.14.66:34015/ using document root <none> and password file <none>
I20260808 16:56:36.718967 10151 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:36.719017 10151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:36.719107 10151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:36.720196 10151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749"
format_stamp: "Formatted at 2026-08-08 16:56:36 on dist-test-slave-1nl6"
server_key: "a684c720b75be43bd50019e750c6cd32"
server_key_iv: "d41b76e40908b8a75bc33622916cc0ce"
server_key_version: "encryptionkey@0"
I20260808 16:56:36.720515 10151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749"
format_stamp: "Formatted at 2026-08-08 16:56:36 on dist-test-slave-1nl6"
server_key: "a684c720b75be43bd50019e750c6cd32"
server_key_iv: "d41b76e40908b8a75bc33622916cc0ce"
server_key_version: "encryptionkey@0"
I20260808 16:56:36.722043 10151 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:56:36.722782 10176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:36.723014 10151 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:56:36.723111 10151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749"
format_stamp: "Formatted at 2026-08-08 16:56:36 on dist-test-slave-1nl6"
server_key: "a684c720b75be43bd50019e750c6cd32"
server_key_iv: "d41b76e40908b8a75bc33622916cc0ce"
server_key_version: "encryptionkey@0"
I20260808 16:56:36.723188 10151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:36.733610 10151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:36.735580 10151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:36.735749 10151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:36.736326 10151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:36.736392 10151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:36.736438 10151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:36.736466 10151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 16:56:36 dist-test-slave-1nl6 krb5kdc[9793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208196, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:36.744079 10151 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:42205
I20260808 16:56:36.744429 10151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:56:36.744475 10344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:42205 every 8 connection(s)
I20260808 16:56:36.746716 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.736731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:43619 (local address 127.8.14.126:40937)
0808 16:56:36.736805 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:36.736809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:36.737050 (+ 241us) server_negotiation.cc:407] Connection header received
0808 16:56:36.737583 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:36.737585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:36.737600 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:36.737682 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:36.738384 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.739251 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.740305 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.740494 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.740575 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.743879 (+ 3304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:36.743881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:36.743882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:36.743890 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:36.745487 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:36.745861 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:36.745862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:36.745862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:36.745918 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:36.746185 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:36.746186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:36.746186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:36.746288 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:36.746347 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:36.746581 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 16:56:36.746645 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 16:56:36.746872 10190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.736708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40937 (local address 127.8.14.66:43619)
0808 16:56:36.736911 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:36.736950 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:36.737429 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:36.737748 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:36.737752 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:36.737979 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:36.738292 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.738307 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.739381 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.739382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.740197 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.740200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.740581 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.740583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.740711 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.741286 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:36.741297 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:36.743774 (+ 2477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:36.745565 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:36.745570 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:36.745571 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:36.745789 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:36.745980 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:36.745980 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:36.745981 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:36.746086 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:36.746356 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:36.746360 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:36.746471 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 16:56:36.746552 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 16:56:36.746675 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":134,"threads_started":1}
I20260808 16:56:36.747345 10345 heartbeater.cc:347] Connected to a master server at 127.8.14.126:40937
I20260808 16:56:36.747432 10345 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:36.747629 10345 heartbeater.cc:511] Master 127.8.14.126:40937 requested a full tablet report, sending...
I20260808 16:56:36.748065 9855 ts_manager.cc:194] Registered new tserver with Master: d67dc6fa7aef4b9aa80345cdcf2a4749 (127.8.14.66:42205)
I20260808 16:56:36.748571 9855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:43619
I20260808 16:56:36.748970 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 10151
I20260808 16:56:36.749073 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:56:36.749266 8249 external_mini_cluster.cc:1428] Setting key 8caeed0a9d71ce11ff2a33cd7aece718
I20260808 16:56:36.755024 10352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.748393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40937 (local address 127.8.14.66:55309)
0808 16:56:36.748517 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:36.748529 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:36.748588 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:36.748717 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:36.748719 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:36.748954 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:36.749028 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.749030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.749967 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.749968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.750778 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.750781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.751215 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.751216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.751307 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.751707 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:36.751713 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:36.752051 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:36.753837 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:36.753838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:36.753839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:36.754003 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:36.754222 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:36.754223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:36.754223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:36.754274 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:36.754636 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:36.754637 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:36.754693 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 16:56:36.754778 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 16:56:36.754872 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
I20260808 16:56:36.755080 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.748473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:55309 (local address 127.8.14.126:40937)
0808 16:56:36.748516 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:36.748518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:36.748572 (+ 54us) server_negotiation.cc:407] Connection header received
0808 16:56:36.748648 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:36.748649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:36.748657 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:36.748706 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:36.749139 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.749826 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.750868 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.751132 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.751212 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.752180 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:36.752182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:36.752182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:36.752190 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:36.753752 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:36.754079 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:36.754080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:36.754081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:36.754152 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:36.754346 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:36.754347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:36.754347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:36.754562 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:36.754644 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:36.754841 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 16:56:36.754950 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 16:56:36.775722 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:40937
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:39639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:36.857465 10357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:36.857667 10357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:36.857689 10357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:36.858815 10357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:36.858858 10357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:36.858877 10357 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:56:36.858979 10357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:56:36.860445 10357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.14.126:40937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:36.860709 10357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:36.860944 10357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:36.865943 10373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:36.865926 10365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:36.865912 10371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:36.866195 10357 server_base.cc:1034] running on GCE node
I20260808 16:56:36.866490 10357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:36.866767 10357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:36.867915 10357 hybrid_clock.cc:648] HybridClock initialized: now 1786208196867900 us; error 31 us; skew 500 ppm
Aug 08 16:56:36 dist-test-slave-1nl6 krb5kdc[9793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208196, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:36.870460 10357 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:56:36.870960 10357 webserver.cc:460] Webserver started at http://127.8.14.67:42483/ using document root <none> and password file <none>
I20260808 16:56:36.871207 10357 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:36.871256 10357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:36.871429 10357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:36.872481 10357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "9f2546443b7c429f82f4931b116f6e15"
format_stamp: "Formatted at 2026-08-08 16:56:36 on dist-test-slave-1nl6"
server_key: "c4d7e7dafd04fe304e161b82a52b1151"
server_key_iv: "96668952bc371f2447950edfbb3a5823"
server_key_version: "encryptionkey@0"
I20260808 16:56:36.872859 10357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "9f2546443b7c429f82f4931b116f6e15"
format_stamp: "Formatted at 2026-08-08 16:56:36 on dist-test-slave-1nl6"
server_key: "c4d7e7dafd04fe304e161b82a52b1151"
server_key_iv: "96668952bc371f2447950edfbb3a5823"
server_key_version: "encryptionkey@0"
I20260808 16:56:36.874472 10357 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:36.875340 10383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:36.875512 10357 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 16:56:36.875625 10357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "9f2546443b7c429f82f4931b116f6e15"
format_stamp: "Formatted at 2026-08-08 16:56:36 on dist-test-slave-1nl6"
server_key: "c4d7e7dafd04fe304e161b82a52b1151"
server_key_iv: "96668952bc371f2447950edfbb3a5823"
server_key_version: "encryptionkey@0"
I20260808 16:56:36.875707 10357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:36.914513 10357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:36.916731 10357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:36.916917 10357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:36.917469 10357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:36.917546 10357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:36.917596 10357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:36.917623 10357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:36.924857 10357 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:34243
I20260808 16:56:36.924984 10545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:34243 every 8 connection(s)
Aug 08 16:56:36 dist-test-slave-1nl6 krb5kdc[9793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208196, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:36.925344 10357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 16:56:36.928326 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.918124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:35235 (local address 127.8.14.126:40937)
0808 16:56:36.918188 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:36.918191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:36.918390 (+ 199us) server_negotiation.cc:407] Connection header received
0808 16:56:36.918997 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:36.918998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:36.919031 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:36.919079 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:36.919893 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.920534 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.922328 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.922705 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.922797 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.925683 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:36.925684 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:36.925684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:36.925693 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:36.927106 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:36.927501 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:36.927502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:36.927502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:36.927554 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:36.927788 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:36.927789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:36.927789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:36.927890 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:36.927941 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:36.928200 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 16:56:36.928265 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 16:56:36.928412 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.918064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40937 (local address 127.8.14.67:35235)
0808 16:56:36.918265 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:36.918302 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:36.918824 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:36.919174 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:36.919177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:36.919551 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:36.919782 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.919788 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.920647 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.920650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.922216 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.922219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.922826 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.922827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.923004 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.923606 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:36.923617 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:36.925569 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:36.927190 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:36.927195 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:36.927197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:36.927416 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:36.927615 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:36.927616 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:36.927616 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:36.927709 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:36.927950 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:36.927954 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:36.928065 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 16:56:36.928146 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 16:56:36.928262 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260808 16:56:36.928900 10546 heartbeater.cc:347] Connected to a master server at 127.8.14.126:40937
I20260808 16:56:36.928988 10546 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:36.929193 10546 heartbeater.cc:511] Master 127.8.14.126:40937 requested a full tablet report, sending...
I20260808 16:56:36.929593 9858 ts_manager.cc:194] Registered new tserver with Master: 9f2546443b7c429f82f4931b116f6e15 (127.8.14.67:34243)
I20260808 16:56:36.929934 9858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:35235
I20260808 16:56:36.930843 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 10357
I20260808 16:56:36.930958 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:56:36.931159 8249 external_mini_cluster.cc:1428] Setting key eefdcdf0d72ed41a643c31a88f013b7b
I20260808 16:56:36.931945 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:56:36.936028 10553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.929974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40937 (local address 127.8.14.67:53977)
0808 16:56:36.930227 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:36.930236 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:36.930319 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:36.930430 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:36.930430 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:36.930616 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:36.930691 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.930693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.931574 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.931575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.932158 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.932160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.932537 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.932538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.932616 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.933068 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:36.933072 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:36.933327 (+ 255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:36.935115 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:36.935116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:36.935116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:36.935279 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:36.935478 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:36.935479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:36.935479 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:36.935521 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:36.935813 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:36.935814 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:36.935846 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 16:56:36.935893 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 16:56:36.935966 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":210,"threads_started":1}
I20260808 16:56:36.936126 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.930052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:53977 (local address 127.8.14.126:40937)
0808 16:56:36.930090 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:36.930092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:36.930310 (+ 218us) server_negotiation.cc:407] Connection header received
0808 16:56:36.930373 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:36.930374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:36.930382 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:36.930424 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:36.930775 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.931466 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.932257 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.932457 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.932538 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.933419 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:36.933420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:36.933420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:36.933427 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:36.935013 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:36.935355 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:36.935356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:36.935357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:36.935412 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:36.935592 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:36.935593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:36.935593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:36.935746 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:36.935817 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:36.935919 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 16:56:36.935998 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 08 16:56:36 dist-test-slave-1nl6 krb5kdc[9793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208196, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 16:56:36 dist-test-slave-1nl6 krb5kdc[9793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208196, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:36.956197 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.949470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39008 (local address 127.8.14.126:40937)
0808 16:56:36.949529 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:36.949531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:36.949540 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:56:36.949612 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:36.949613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:36.949626 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:36.949711 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:36.950243 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.950702 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.951564 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.951746 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.951799 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.953865 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:36.953867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:36.953867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:36.953874 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:36.955108 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:36.955543 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:36.955543 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:36.955544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:36.955594 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:36.955770 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:36.955771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:36.955771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:36.955865 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:36.955937 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:36.956074 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 16:56:36.956135 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 16:56:36.957749 9858 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:56:36.959444 9858 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:56:36.966133 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.961792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:40855 (local address 127.0.0.1:52216)
0808 16:56:36.962034 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:36.962043 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:36.962084 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:36.962601 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:36.962602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:36.962604 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:36.962708 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.962710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.964046 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.964047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.965143 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.965145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.965833 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.965834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.965928 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.965942 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 16:56:36.966010 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 16:56:36.966064 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":60,"threads_started":1}
I20260808 16:56:36.966462 10576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.962240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52216 (local address 127.8.14.65:40855)
0808 16:56:36.962449 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:36.962453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:36.962462 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:56:36.962504 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:36.962506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:36.962518 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:36.962588 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:36.962803 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.963950 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.965232 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.965710 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.965830 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.965895 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:36.966333 (+ 438us) server_negotiation.cc:299] Negotiation successful
0808 16:56:36.966373 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":98,"threads_started":1}
I20260808 16:56:36.967044 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.961785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:34243 (local address 127.0.0.1:43214)
0808 16:56:36.961968 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:36.962007 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:36.962078 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:36.962431 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:36.962439 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:36.962441 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:36.962726 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.962729 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.963819 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.963820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.965143 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.965145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.966759 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.966760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.966821 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.966825 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:56:36.966870 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 16:56:36.966963 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":8,"thread_start_us":93,"threads_started":1}
I20260808 16:56:36.967176 10041 tablet_service.cc:1467] Processing CreateTablet for tablet 93089e3e0f7545bab122005cd3b21b11 (DEFAULT_TABLE table=test-table [id=457fefed3ea748d8b052ae42b8a79590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:36.967306 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.961799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:42205 (local address 127.0.0.1:47348)
0808 16:56:36.962178 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:36.962186 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:36.962222 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:36.962514 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:36.962515 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:36.962517 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:36.962736 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.962738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.965249 (+ 2511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.965250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.965939 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:36.965941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.967084 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.967085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:36.967161 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.967167 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 16:56:36.967202 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 16:56:36.967241 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":97,"threads_started":1}
I20260808 16:56:36.967336 10575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.962029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43214 (local address 127.8.14.67:34243)
0808 16:56:36.962198 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:36.962203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:36.962214 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:56:36.962256 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:36.962258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:36.962276 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:36.962816 (+ 540us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:36.962873 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.963716 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.965933 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.966687 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.966760 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.967212 (+ 452us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:36.967233 (+ 21us) server_negotiation.cc:299] Negotiation successful
0808 16:56:36.967263 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260808 16:56:36.967593 10041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93089e3e0f7545bab122005cd3b21b11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:36.967772 10574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:36.961878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47348 (local address 127.8.14.66:42205)
0808 16:56:36.962320 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:36.962324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:36.962332 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:56:36.962398 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:36.962400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:36.962416 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:36.962499 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:36.962822 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.963944 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.966053 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:36.966980 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:36.967518 (+ 538us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:36.967582 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:36.967609 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 16:56:36.967649 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":77,"threads_started":1}
I20260808 16:56:36.967988 10425 tablet_service.cc:1467] Processing CreateTablet for tablet 93089e3e0f7545bab122005cd3b21b11 (DEFAULT_TABLE table=test-table [id=457fefed3ea748d8b052ae42b8a79590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:36.968268 10425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93089e3e0f7545bab122005cd3b21b11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:36.968421 10248 tablet_service.cc:1467] Processing CreateTablet for tablet 93089e3e0f7545bab122005cd3b21b11 (DEFAULT_TABLE table=test-table [id=457fefed3ea748d8b052ae42b8a79590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:36.968763 10248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93089e3e0f7545bab122005cd3b21b11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:36.970726 10578 tablet_bootstrap.cc:492] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15: Bootstrap starting.
I20260808 16:56:36.971580 10579 tablet_bootstrap.cc:492] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853: Bootstrap starting.
I20260808 16:56:36.971776 10578 tablet_bootstrap.cc:654] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:36.972143 10578 log.cc:824] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:36.972268 10579 tablet_bootstrap.cc:654] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:36.972648 10581 tablet_bootstrap.cc:492] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749: Bootstrap starting.
I20260808 16:56:36.972659 10579 log.cc:824] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:36.973050 10578 tablet_bootstrap.cc:492] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15: No bootstrap required, opened a new log
I20260808 16:56:36.973142 10578 ts_tablet_manager.cc:1397] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:56:36.973304 10581 tablet_bootstrap.cc:654] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:36.973472 10579 tablet_bootstrap.cc:492] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853: No bootstrap required, opened a new log
I20260808 16:56:36.973548 10579 ts_tablet_manager.cc:1397] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:56:36.973649 10581 log.cc:824] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:36.974503 10581 tablet_bootstrap.cc:492] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749: No bootstrap required, opened a new log
I20260808 16:56:36.974577 10581 ts_tablet_manager.cc:1397] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:56:36.975214 10578 raft_consensus.cc:348] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } }
I20260808 16:56:36.975380 10578 raft_consensus.cc:374] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:36.975425 10578 raft_consensus.cc:729] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f2546443b7c429f82f4931b116f6e15, State: Initialized, Role: FOLLOWER
I20260808 16:56:36.975548 10578 consensus_queue.cc:260] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } }
I20260808 16:56:36.975826 10578 ts_tablet_manager.cc:1428] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260808 16:56:36.975837 10546 heartbeater.cc:503] Master 127.8.14.126:40937 was elected leader, sending a full tablet report...
I20260808 16:56:36.975775 10579 raft_consensus.cc:348] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } }
I20260808 16:56:36.975926 10579 raft_consensus.cc:374] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:36.975960 10579 raft_consensus.cc:729] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3985510e5db45529e0de98fa65a1853, State: Initialized, Role: FOLLOWER
I20260808 16:56:36.976065 10579 consensus_queue.cc:260] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } }
I20260808 16:56:36.976331 10579 ts_tablet_manager.cc:1428] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:56:36.976461 10140 heartbeater.cc:503] Master 127.8.14.126:40937 was elected leader, sending a full tablet report...
I20260808 16:56:36.976557 10581 raft_consensus.cc:348] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } }
I20260808 16:56:36.976734 10581 raft_consensus.cc:374] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:36.976783 10581 raft_consensus.cc:729] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d67dc6fa7aef4b9aa80345cdcf2a4749, State: Initialized, Role: FOLLOWER
I20260808 16:56:36.976890 10581 consensus_queue.cc:260] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } }
I20260808 16:56:36.977190 10581 ts_tablet_manager.cc:1428] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:56:36.977294 10345 heartbeater.cc:503] Master 127.8.14.126:40937 was elected leader, sending a full tablet report...
W20260808 16:56:36.995033 10347 tablet.cc:2369] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:56:37.090209 10141 tablet.cc:2369] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:56:37.176061 10547 tablet.cc:2369] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:56:37.241854 10588 raft_consensus.cc:482] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:56:37.241997 10588 raft_consensus.cc:504] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } }
I20260808 16:56:37.242347 10588 leader_election.cc:288] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f2546443b7c429f82f4931b116f6e15 (127.8.14.67:34243), d67dc6fa7aef4b9aa80345cdcf2a4749 (127.8.14.66:42205)
I20260808 16:56:37.245517 10575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:37.242688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:54433 (local address 127.8.14.67:34243)
0808 16:56:37.242754 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:37.242756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:37.242766 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:56:37.242797 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:37.242798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:37.242810 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:37.242914 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:37.243066 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.243568 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.244663 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.245158 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.245247 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:37.245256 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:37.245410 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 16:56:37.245446 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 16:56:37.245535 10591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:37.242529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:34243 (local address 127.8.14.65:54433)
0808 16:56:37.242650 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:37.242662 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:37.242726 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:37.242892 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:37.242893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:37.242895 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:37.242984 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:37.242985 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.243661 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.243662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:37.244492 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:37.244495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.245250 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.245251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:37.245327 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:37.245330 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:56:37.245404 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 16:56:37.245446 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":68,"threads_started":1}
I20260808 16:56:37.245535 10592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:37.242587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:42205 (local address 127.8.14.65:39153)
0808 16:56:37.242745 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:37.242755 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:37.242850 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:37.243028 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:37.243029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:37.243031 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:37.243100 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:37.243101 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.243732 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.243733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:37.244460 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:37.244462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.245212 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.245213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:37.245285 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:37.245289 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:56:37.245369 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 16:56:37.245408 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260808 16:56:37.245596 10574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:37.242657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:39153 (local address 127.8.14.66:42205)
0808 16:56:37.242707 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:37.242709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:37.242837 (+ 128us) server_negotiation.cc:407] Connection header received
0808 16:56:37.242939 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:37.242944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:37.242956 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:37.243023 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:37.243185 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.243634 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.244581 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.245123 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.245211 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:37.245223 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:37.245472 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 16:56:37.245518 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:56:37.246027 10481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93089e3e0f7545bab122005cd3b21b11" candidate_uuid: "d3985510e5db45529e0de98fa65a1853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f2546443b7c429f82f4931b116f6e15" is_pre_election: true
I20260808 16:56:37.246029 10278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93089e3e0f7545bab122005cd3b21b11" candidate_uuid: "d3985510e5db45529e0de98fa65a1853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" is_pre_election: true
I20260808 16:56:37.246189 10278 raft_consensus.cc:2454] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3985510e5db45529e0de98fa65a1853 in term 0.
I20260808 16:56:37.246189 10481 raft_consensus.cc:2454] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3985510e5db45529e0de98fa65a1853 in term 0.
I20260808 16:56:37.246434 9977 leader_election.cc:302] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f2546443b7c429f82f4931b116f6e15, d3985510e5db45529e0de98fa65a1853; no voters:
I20260808 16:56:37.246569 10588 raft_consensus.cc:2781] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:56:37.246628 10588 raft_consensus.cc:482] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:56:37.246652 10588 raft_consensus.cc:3037] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:37.247517 10588 raft_consensus.cc:504] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } }
I20260808 16:56:37.247673 10588 leader_election.cc:288] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [CANDIDATE]: Term 1 election: Requested vote from peers 9f2546443b7c429f82f4931b116f6e15 (127.8.14.67:34243), d67dc6fa7aef4b9aa80345cdcf2a4749 (127.8.14.66:42205)
I20260808 16:56:37.247829 10481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93089e3e0f7545bab122005cd3b21b11" candidate_uuid: "d3985510e5db45529e0de98fa65a1853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f2546443b7c429f82f4931b116f6e15"
I20260808 16:56:37.247854 10278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93089e3e0f7545bab122005cd3b21b11" candidate_uuid: "d3985510e5db45529e0de98fa65a1853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749"
I20260808 16:56:37.247937 10481 raft_consensus.cc:3037] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:37.247939 10278 raft_consensus.cc:3037] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:37.248842 10481 raft_consensus.cc:2454] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3985510e5db45529e0de98fa65a1853 in term 1.
I20260808 16:56:37.248893 10278 raft_consensus.cc:2454] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3985510e5db45529e0de98fa65a1853 in term 1.
I20260808 16:56:37.249043 9977 leader_election.cc:302] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f2546443b7c429f82f4931b116f6e15, d3985510e5db45529e0de98fa65a1853; no voters:
I20260808 16:56:37.249166 10588 raft_consensus.cc:2781] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:37.249306 10588 raft_consensus.cc:686] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [term 1 LEADER]: Becoming Leader. State: Replica: d3985510e5db45529e0de98fa65a1853, State: Running, Role: LEADER
I20260808 16:56:37.249414 10588 consensus_queue.cc:237] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } }
I20260808 16:56:37.250166 9855 catalog_manager.cc:5495] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3985510e5db45529e0de98fa65a1853 (127.8.14.65). New cstate: current_term: 1 leader_uuid: "d3985510e5db45529e0de98fa65a1853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } health_report { overall_health: HEALTHY } } }
I20260808 16:56:37.320698 10576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:37.317947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52218 (local address 127.8.14.65:40855)
0808 16:56:37.318034 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:37.318036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:37.318047 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:56:37.318129 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:37.318130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:37.318155 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:37.318226 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:37.318411 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.318955 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.319764 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.319979 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.320066 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:37.320319 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:37.320431 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:37.320531 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:37.320588 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 16:56:37.320610 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 16:56:37.322538 10278 raft_consensus.cc:1261] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 1 FOLLOWER]: Refusing update from remote peer d3985510e5db45529e0de98fa65a1853: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:37.322554 10481 raft_consensus.cc:1261] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15 [term 1 FOLLOWER]: Refusing update from remote peer d3985510e5db45529e0de98fa65a1853: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:37.322924 10588 consensus_queue.cc:1035] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:37.323076 10593 consensus_queue.cc:1035] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:37.326683 10597 mvcc.cc:205] Tried to move back new op lower bound from 7316308776230375424 to 7316308775934152704. Current Snapshot: MvccSnapshot[applied={T|T < 7316308776230375424}]
I20260808 16:56:37.326856 10598 mvcc.cc:205] Tried to move back new op lower bound from 7316308776230375424 to 7316308775934152704. Current Snapshot: MvccSnapshot[applied={T|T < 7316308776230375424}]
I20260808 16:56:37.329895 9855 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:56:37.330049 9855 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:56:37.330965 9855 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b: Sending DeleteTablet for 3 replicas of tablet 93089e3e0f7545bab122005cd3b21b11
I20260808 16:56:37.331303 10425 tablet_service.cc:1514] Processing DeleteTablet for tablet 93089e3e0f7545bab122005cd3b21b11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:37 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:43214
I20260808 16:56:37.331359 10248 tablet_service.cc:1514] Processing DeleteTablet for tablet 93089e3e0f7545bab122005cd3b21b11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:37 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:47348
I20260808 16:56:37.331539 10616 tablet_replica.cc:330] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15: stopping tablet replica
I20260808 16:56:37.331553 10041 tablet_service.cc:1514] Processing DeleteTablet for tablet 93089e3e0f7545bab122005cd3b21b11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:37 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:52216
I20260808 16:56:37.331635 10616 raft_consensus.cc:2229] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:56:37.331574 9855 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:56:37.331708 9855 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:56:37.331743 10616 raft_consensus.cc:2258] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:37.332165 10619 tablet_replica.cc:330] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853: stopping tablet replica
I20260808 16:56:37.332324 10618 tablet_replica.cc:330] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749: stopping tablet replica
I20260808 16:56:37.332427 10616 ts_tablet_manager.cc:1905] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:37.332458 10618 raft_consensus.cc:2229] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:56:37.332577 10619 raft_consensus.cc:2229] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:56:37.332744 10618 raft_consensus.cc:2258] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:37.332963 10619 raft_consensus.cc:2258] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:37.333153 10041 tablet_service.cc:1467] Processing CreateTablet for tablet a6d7adb55412482cbe43a89cc2b97f18 (DEFAULT_TABLE table=test-table [id=b55ca5277f824d45957aab4606d90e9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:37.333292 10041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d7adb55412482cbe43a89cc2b97f18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:37.333575 10619 ts_tablet_manager.cc:1905] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:37.334580 10616 ts_tablet_manager.cc:1918] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:56:37.334649 10616 log.cc:1196] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/93089e3e0f7545bab122005cd3b21b11
I20260808 16:56:37.334758 10579 tablet_bootstrap.cc:492] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853: Bootstrap starting.
I20260808 16:56:37.335198 10579 tablet_bootstrap.cc:654] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:37.335263 10618 ts_tablet_manager.cc:1905] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:37.335286 10619 ts_tablet_manager.cc:1918] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:56:37.335345 10619 log.cc:1196] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/wal/wals/93089e3e0f7545bab122005cd3b21b11
I20260808 16:56:37.335594 10619 ts_tablet_manager.cc:1939] T 93089e3e0f7545bab122005cd3b21b11 P d3985510e5db45529e0de98fa65a1853: Deleting consensus metadata
I20260808 16:56:37.332957 10248 tablet_service.cc:1467] Processing CreateTablet for tablet a6d7adb55412482cbe43a89cc2b97f18 (DEFAULT_TABLE table=test-table [id=b55ca5277f824d45957aab4606d90e9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:37.335992 10248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d7adb55412482cbe43a89cc2b97f18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:37.334935 10616 ts_tablet_manager.cc:1939] T 93089e3e0f7545bab122005cd3b21b11 P 9f2546443b7c429f82f4931b116f6e15: Deleting consensus metadata
I20260808 16:56:37.336786 9834 catalog_manager.cc:4841] TS d3985510e5db45529e0de98fa65a1853 (127.8.14.65:40855): tablet 93089e3e0f7545bab122005cd3b21b11 (table test-table [id=457fefed3ea748d8b052ae42b8a79590]) successfully deleted
I20260808 16:56:37.337201 10425 tablet_service.cc:1467] Processing CreateTablet for tablet a6d7adb55412482cbe43a89cc2b97f18 (DEFAULT_TABLE table=test-table [id=b55ca5277f824d45957aab4606d90e9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:37.337329 10425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d7adb55412482cbe43a89cc2b97f18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:37.337420 10581 tablet_bootstrap.cc:492] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749: Bootstrap starting.
I20260808 16:56:37.337522 10618 ts_tablet_manager.cc:1918] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:56:37.337591 10618 log.cc:1196] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/93089e3e0f7545bab122005cd3b21b11
I20260808 16:56:37.337821 10581 tablet_bootstrap.cc:654] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:37.337875 10618 ts_tablet_manager.cc:1939] T 93089e3e0f7545bab122005cd3b21b11 P d67dc6fa7aef4b9aa80345cdcf2a4749: Deleting consensus metadata
I20260808 16:56:37.338495 10581 tablet_bootstrap.cc:492] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749: No bootstrap required, opened a new log
I20260808 16:56:37.338529 10578 tablet_bootstrap.cc:492] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15: Bootstrap starting.
I20260808 16:56:37.338547 10581 ts_tablet_manager.cc:1397] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:37.338570 9835 catalog_manager.cc:4841] TS d67dc6fa7aef4b9aa80345cdcf2a4749 (127.8.14.66:42205): tablet 93089e3e0f7545bab122005cd3b21b11 (table test-table [id=457fefed3ea748d8b052ae42b8a79590]) successfully deleted
I20260808 16:56:37.338886 9836 catalog_manager.cc:4841] TS 9f2546443b7c429f82f4931b116f6e15 (127.8.14.67:34243): tablet 93089e3e0f7545bab122005cd3b21b11 (table test-table [id=457fefed3ea748d8b052ae42b8a79590]) successfully deleted
I20260808 16:56:37.338866 10581 raft_consensus.cc:348] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.338989 10581 raft_consensus.cc:374] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:37.338990 10578 tablet_bootstrap.cc:654] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:37.339016 10581 raft_consensus.cc:729] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d67dc6fa7aef4b9aa80345cdcf2a4749, State: Initialized, Role: FOLLOWER
I20260808 16:56:37.339066 10581 consensus_queue.cc:260] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.339187 10581 ts_tablet_manager.cc:1428] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:37.339725 10578 tablet_bootstrap.cc:492] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15: No bootstrap required, opened a new log
I20260808 16:56:37.339777 10578 ts_tablet_manager.cc:1397] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 16:56:37.339877 10579 tablet_bootstrap.cc:492] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853: No bootstrap required, opened a new log
I20260808 16:56:37.339923 10579 ts_tablet_manager.cc:1397] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 16:56:37.339942 10578 raft_consensus.cc:348] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.339994 10578 raft_consensus.cc:374] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:37.340008 10578 raft_consensus.cc:729] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f2546443b7c429f82f4931b116f6e15, State: Initialized, Role: FOLLOWER
I20260808 16:56:37.340042 10578 consensus_queue.cc:260] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.340061 10579 raft_consensus.cc:348] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.340135 10579 raft_consensus.cc:374] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:37.340147 10578 ts_tablet_manager.cc:1428] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:37.340157 10579 raft_consensus.cc:729] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3985510e5db45529e0de98fa65a1853, State: Initialized, Role: FOLLOWER
I20260808 16:56:37.340197 10579 consensus_queue.cc:260] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.340299 10579 ts_tablet_manager.cc:1428] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:37.621323 10608 raft_consensus.cc:482] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:56:37.621430 10608 raft_consensus.cc:504] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.621652 10608 leader_election.cc:288] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d67dc6fa7aef4b9aa80345cdcf2a4749 (127.8.14.66:42205), 9f2546443b7c429f82f4931b116f6e15 (127.8.14.67:34243)
I20260808 16:56:37.621894 10481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d7adb55412482cbe43a89cc2b97f18" candidate_uuid: "d3985510e5db45529e0de98fa65a1853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f2546443b7c429f82f4931b116f6e15" is_pre_election: true
I20260808 16:56:37.621894 10278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d7adb55412482cbe43a89cc2b97f18" candidate_uuid: "d3985510e5db45529e0de98fa65a1853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" is_pre_election: true
I20260808 16:56:37.622009 10481 raft_consensus.cc:2454] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3985510e5db45529e0de98fa65a1853 in term 0.
I20260808 16:56:37.622009 10278 raft_consensus.cc:2454] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3985510e5db45529e0de98fa65a1853 in term 0.
I20260808 16:56:37.622166 9977 leader_election.cc:302] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f2546443b7c429f82f4931b116f6e15, d3985510e5db45529e0de98fa65a1853; no voters:
I20260808 16:56:37.622256 10608 raft_consensus.cc:2781] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:56:37.622314 10608 raft_consensus.cc:482] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:56:37.622346 10608 raft_consensus.cc:3037] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:37.623276 10608 raft_consensus.cc:504] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.623435 10608 leader_election.cc:288] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [CANDIDATE]: Term 1 election: Requested vote from peers d67dc6fa7aef4b9aa80345cdcf2a4749 (127.8.14.66:42205), 9f2546443b7c429f82f4931b116f6e15 (127.8.14.67:34243)
I20260808 16:56:37.623644 10278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d7adb55412482cbe43a89cc2b97f18" candidate_uuid: "d3985510e5db45529e0de98fa65a1853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749"
I20260808 16:56:37.623654 10481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d7adb55412482cbe43a89cc2b97f18" candidate_uuid: "d3985510e5db45529e0de98fa65a1853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f2546443b7c429f82f4931b116f6e15"
I20260808 16:56:37.623750 10481 raft_consensus.cc:3037] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:37.623754 10278 raft_consensus.cc:3037] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:37.624389 10481 raft_consensus.cc:2454] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3985510e5db45529e0de98fa65a1853 in term 1.
I20260808 16:56:37.624415 10278 raft_consensus.cc:2454] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3985510e5db45529e0de98fa65a1853 in term 1.
I20260808 16:56:37.624601 9976 leader_election.cc:302] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3985510e5db45529e0de98fa65a1853, d67dc6fa7aef4b9aa80345cdcf2a4749; no voters:
I20260808 16:56:37.624717 10608 raft_consensus.cc:2781] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:37.624787 10608 raft_consensus.cc:686] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [term 1 LEADER]: Becoming Leader. State: Replica: d3985510e5db45529e0de98fa65a1853, State: Running, Role: LEADER
I20260808 16:56:37.624848 10608 consensus_queue.cc:237] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.625459 9853 catalog_manager.cc:5495] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3985510e5db45529e0de98fa65a1853 (127.8.14.65). New cstate: current_term: 1 leader_uuid: "d3985510e5db45529e0de98fa65a1853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } health_report { overall_health: UNKNOWN } } }
Aug 08 16:56:37 dist-test-slave-1nl6 krb5kdc[9793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208196, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:37.719270 10633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:37.707659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40937 (local address 127.0.0.1:59292)
0808 16:56:37.707871 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:37.707887 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:37.707969 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:37.708186 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:37.708187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:37.708521 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:37.708565 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:37.708568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.709253 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.709254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:37.709772 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:37.709773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.710132 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.710133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:37.710215 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:37.715214 (+ 4999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:37.715219 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:37.716622 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:37.718068 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:37.718075 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:37.718077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:37.718341 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:37.718533 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:37.718534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:37.718534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:37.718595 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:37.718849 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:37.718855 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:37.718936 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 16:56:37.719003 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 16:56:37.719103 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":118,"threads_started":1}
I20260808 16:56:37.719389 10634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:37.707741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59292 (local address 127.8.14.126:40937)
0808 16:56:37.707915 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:37.707918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:37.707946 (+ 28us) server_negotiation.cc:407] Connection header received
0808 16:56:37.708063 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:37.708064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:37.708075 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:37.708161 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:37.708673 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.709167 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.709862 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.710057 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.710134 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:37.716718 (+ 6584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:37.716719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:37.716720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:37.716727 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:37.717995 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:37.718413 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:37.718414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:37.718414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:37.718472 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:37.718651 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:37.718652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:37.718652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:37.718786 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:37.718851 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:37.719068 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 16:56:37.719165 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260808 16:56:37.721537 9853 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:59292:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 16:56:37.721764 9853 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:56:37.723335 10041 tablet_service.cc:1467] Processing CreateTablet for tablet b587035193804977aa3b058e4dc6b354 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a7bf9a69677b465ca4c0577a6fc3f177]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:56:37.723367 10248 tablet_service.cc:1467] Processing CreateTablet for tablet b587035193804977aa3b058e4dc6b354 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a7bf9a69677b465ca4c0577a6fc3f177]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:56:37.723371 10425 tablet_service.cc:1467] Processing CreateTablet for tablet b587035193804977aa3b058e4dc6b354 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a7bf9a69677b465ca4c0577a6fc3f177]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:56:37.723515 10041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b587035193804977aa3b058e4dc6b354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:37.723515 10425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b587035193804977aa3b058e4dc6b354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:37.723532 10248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b587035193804977aa3b058e4dc6b354. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:37.725836 10581 tablet_bootstrap.cc:492] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749: Bootstrap starting.
I20260808 16:56:37.726259 10581 tablet_bootstrap.cc:654] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:37.726649 10578 tablet_bootstrap.cc:492] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15: Bootstrap starting.
I20260808 16:56:37.727105 10578 tablet_bootstrap.cc:654] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:37.727376 10579 tablet_bootstrap.cc:492] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853: Bootstrap starting.
I20260808 16:56:37.727685 10579 tablet_bootstrap.cc:654] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:37.727718 10578 tablet_bootstrap.cc:492] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15: No bootstrap required, opened a new log
I20260808 16:56:37.727756 10578 ts_tablet_manager.cc:1397] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 16:56:37.727943 10578 raft_consensus.cc:348] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.728024 10578 raft_consensus.cc:374] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:37.728046 10578 raft_consensus.cc:729] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f2546443b7c429f82f4931b116f6e15, State: Initialized, Role: FOLLOWER
I20260808 16:56:37.728098 10578 consensus_queue.cc:260] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.728132 10581 tablet_bootstrap.cc:492] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749: No bootstrap required, opened a new log
I20260808 16:56:37.728178 10581 ts_tablet_manager.cc:1397] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:56:37.728223 10578 ts_tablet_manager.cc:1428] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 16:56:37.728288 10585 tablet_replica.cc:439] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } }
I20260808 16:56:37.728331 10581 raft_consensus.cc:348] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.728399 10581 raft_consensus.cc:374] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:37.728403 10585 tablet_replica.cc:442] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:56:37.728436 10581 raft_consensus.cc:729] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d67dc6fa7aef4b9aa80345cdcf2a4749, State: Initialized, Role: FOLLOWER
I20260808 16:56:37.728510 10581 consensus_queue.cc:260] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.728626 10581 ts_tablet_manager.cc:1428] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:37.728686 10589 tablet_replica.cc:439] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } }
I20260808 16:56:37.728785 10589 tablet_replica.cc:442] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:56:37.729004 10579 tablet_bootstrap.cc:492] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853: No bootstrap required, opened a new log
I20260808 16:56:37.729056 10579 ts_tablet_manager.cc:1397] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:56:37.729166 10579 raft_consensus.cc:348] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.729238 10579 raft_consensus.cc:374] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:37.729259 10579 raft_consensus.cc:729] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3985510e5db45529e0de98fa65a1853, State: Initialized, Role: FOLLOWER
I20260808 16:56:37.729312 10579 consensus_queue.cc:260] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.729408 10579 ts_tablet_manager.cc:1428] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:37.729421 10608 tablet_replica.cc:439] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } }
I20260808 16:56:37.729488 10608 tablet_replica.cc:442] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:56:37.762387 10608 raft_consensus.cc:482] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:56:37.762476 10608 raft_consensus.cc:504] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.762651 10608 leader_election.cc:288] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d67dc6fa7aef4b9aa80345cdcf2a4749 (127.8.14.66:42205), 9f2546443b7c429f82f4931b116f6e15 (127.8.14.67:34243)
I20260808 16:56:37.762856 10278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b587035193804977aa3b058e4dc6b354" candidate_uuid: "d3985510e5db45529e0de98fa65a1853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" is_pre_election: true
I20260808 16:56:37.762895 10481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b587035193804977aa3b058e4dc6b354" candidate_uuid: "d3985510e5db45529e0de98fa65a1853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f2546443b7c429f82f4931b116f6e15" is_pre_election: true
I20260808 16:56:37.763013 10278 raft_consensus.cc:2454] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3985510e5db45529e0de98fa65a1853 in term 0.
I20260808 16:56:37.763037 10481 raft_consensus.cc:2454] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3985510e5db45529e0de98fa65a1853 in term 0.
I20260808 16:56:37.763221 9977 leader_election.cc:302] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f2546443b7c429f82f4931b116f6e15, d3985510e5db45529e0de98fa65a1853; no voters:
I20260808 16:56:37.763360 10608 raft_consensus.cc:2781] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:56:37.763401 10608 raft_consensus.cc:482] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:56:37.763423 10608 raft_consensus.cc:3037] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:37.764099 10608 raft_consensus.cc:504] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.764240 10608 leader_election.cc:288] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [CANDIDATE]: Term 1 election: Requested vote from peers d67dc6fa7aef4b9aa80345cdcf2a4749 (127.8.14.66:42205), 9f2546443b7c429f82f4931b116f6e15 (127.8.14.67:34243)
I20260808 16:56:37.764415 10481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b587035193804977aa3b058e4dc6b354" candidate_uuid: "d3985510e5db45529e0de98fa65a1853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f2546443b7c429f82f4931b116f6e15"
I20260808 16:56:37.764472 10278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b587035193804977aa3b058e4dc6b354" candidate_uuid: "d3985510e5db45529e0de98fa65a1853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749"
I20260808 16:56:37.764508 10481 raft_consensus.cc:3037] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:37.764539 10278 raft_consensus.cc:3037] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:37.765094 10278 raft_consensus.cc:2454] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3985510e5db45529e0de98fa65a1853 in term 1.
I20260808 16:56:37.765120 10481 raft_consensus.cc:2454] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3985510e5db45529e0de98fa65a1853 in term 1.
I20260808 16:56:37.765290 9977 leader_election.cc:302] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f2546443b7c429f82f4931b116f6e15, d3985510e5db45529e0de98fa65a1853; no voters:
I20260808 16:56:37.765408 10608 raft_consensus.cc:2781] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:37.765463 10608 raft_consensus.cc:686] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [term 1 LEADER]: Becoming Leader. State: Replica: d3985510e5db45529e0de98fa65a1853, State: Running, Role: LEADER
I20260808 16:56:37.765530 10608 consensus_queue.cc:237] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } }
I20260808 16:56:37.765800 10588 tablet_replica.cc:439] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853: TxnStatusTablet state changed. Reason: New leader d3985510e5db45529e0de98fa65a1853. Latest consensus state: current_term: 1 leader_uuid: "d3985510e5db45529e0de98fa65a1853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } }
I20260808 16:56:37.765900 10588 tablet_replica.cc:442] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:56:37.766058 10645 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 16:56:37.766175 10645 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 16:56:37.766110 9858 catalog_manager.cc:5495] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3985510e5db45529e0de98fa65a1853 (127.8.14.65). New cstate: current_term: 1 leader_uuid: "d3985510e5db45529e0de98fa65a1853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } health_report { overall_health: UNKNOWN } } }
I20260808 16:56:37.773610 10576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:37.771276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49070 (local address 127.8.14.65:40855)
0808 16:56:37.771332 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:37.771334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:37.771343 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:56:37.771425 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:37.771426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:37.771437 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:37.771492 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:37.771626 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.772108 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.772883 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.773088 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.773167 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:37.773363 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:37.773395 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:37.773442 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:37.773524 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 16:56:37.773545 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:56:37.773670 10633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:37.771208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:40855 (local address 127.0.0.1:49070)
0808 16:56:37.771287 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:37.771308 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:37.771362 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:37.771500 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:37.771501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:37.771502 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:56:37.771551 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:37.771553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.772219 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.772220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:37.772793 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:37.772794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.773166 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.773167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:37.773231 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:37.773258 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:56:37.773459 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:56:37.773467 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 16:56:37.773519 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 16:56:37.773587 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 16:56:37.774601 10481 raft_consensus.cc:1261] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15 [term 1 FOLLOWER]: Refusing update from remote peer d3985510e5db45529e0de98fa65a1853: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:37.774614 10278 raft_consensus.cc:1261] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 1 FOLLOWER]: Refusing update from remote peer d3985510e5db45529e0de98fa65a1853: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:37.774806 10588 consensus_queue.cc:1035] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:37.774914 10608 consensus_queue.cc:1035] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:37.775466 10585 tablet_replica.cc:439] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15: TxnStatusTablet state changed. Reason: New leader d3985510e5db45529e0de98fa65a1853. Latest consensus state: current_term: 1 leader_uuid: "d3985510e5db45529e0de98fa65a1853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } }
I20260808 16:56:37.775547 10585 tablet_replica.cc:442] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:56:37.775486 10589 tablet_replica.cc:439] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749: TxnStatusTablet state changed. Reason: New leader d3985510e5db45529e0de98fa65a1853. Latest consensus state: current_term: 1 leader_uuid: "d3985510e5db45529e0de98fa65a1853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } }
I20260808 16:56:37.775578 10589 tablet_replica.cc:442] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:56:37.776234 10608 tablet_replica.cc:439] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d3985510e5db45529e0de98fa65a1853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } }
I20260808 16:56:37.776340 10608 tablet_replica.cc:442] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:56:37.776398 10608 tablet_replica.cc:439] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d3985510e5db45529e0de98fa65a1853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } }
I20260808 16:56:37.776466 10608 tablet_replica.cc:442] T b587035193804977aa3b058e4dc6b354 P d3985510e5db45529e0de98fa65a1853: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:56:37.776515 10601 mvcc.cc:205] Tried to move back new op lower bound from 7316308778083381248 to 7316308778047967232. Current Snapshot: MvccSnapshot[applied={T|T < 7316308778083381248}]
I20260808 16:56:37.776603 10585 tablet_replica.cc:439] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d3985510e5db45529e0de98fa65a1853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } }
I20260808 16:56:37.776691 10585 tablet_replica.cc:442] T b587035193804977aa3b058e4dc6b354 P 9f2546443b7c429f82f4931b116f6e15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:56:37.777837 10589 tablet_replica.cc:439] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d3985510e5db45529e0de98fa65a1853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3985510e5db45529e0de98fa65a1853" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40855 } } peers { permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 } } peers { permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 } } }
I20260808 16:56:37.777920 10589 tablet_replica.cc:442] T b587035193804977aa3b058e4dc6b354 P d67dc6fa7aef4b9aa80345cdcf2a4749: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:56:37.783391 10576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:37.780774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:36605 (local address 127.8.14.65:40855)
0808 16:56:37.780812 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:37.780814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:37.780974 (+ 160us) server_negotiation.cc:407] Connection header received
0808 16:56:37.781029 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:37.781030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:37.781038 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:37.781078 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:37.781189 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.781644 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.782713 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.782883 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.783004 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:37.783171 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:37.783199 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:37.783234 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:37.783326 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 16:56:37.783345 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":18}
I20260808 16:56:37.783488 10658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:37.780698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:40855 (local address 127.8.14.65:36605)
0808 16:56:37.780879 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:37.780889 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:37.780973 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:37.781091 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:37.781091 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:37.781093 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:56:37.781137 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:37.781138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.781713 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.781714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:37.782632 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:37.782634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.783008 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.783009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:37.783068 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:37.783084 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:56:37.783263 (+ 179us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:56:37.783264 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 16:56:37.783324 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 16:56:37.783375 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":89,"threads_started":1}
I20260808 16:56:37.785467 10481 raft_consensus.cc:1261] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15 [term 1 FOLLOWER]: Refusing update from remote peer d3985510e5db45529e0de98fa65a1853: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:37.785480 10278 raft_consensus.cc:1261] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 1 FOLLOWER]: Refusing update from remote peer d3985510e5db45529e0de98fa65a1853: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:37.785683 10588 consensus_queue.cc:1035] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d67dc6fa7aef4b9aa80345cdcf2a4749" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:37.785827 10588 consensus_queue.cc:1035] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f2546443b7c429f82f4931b116f6e15" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 34243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:37.796968 9858 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:56:37.797071 9858 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:56:37.797739 9858 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d8d4f59aaf93434e93ab21373ff99c2b: Sending DeleteTablet for 3 replicas of tablet a6d7adb55412482cbe43a89cc2b97f18
I20260808 16:56:37.797971 10041 tablet_service.cc:1514] Processing DeleteTablet for tablet a6d7adb55412482cbe43a89cc2b97f18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:37 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:52216
I20260808 16:56:37.797979 10248 tablet_service.cc:1514] Processing DeleteTablet for tablet a6d7adb55412482cbe43a89cc2b97f18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:37 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:47348
I20260808 16:56:37.798065 10619 tablet_replica.cc:330] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853: stopping tablet replica
I20260808 16:56:37.798061 10425 tablet_service.cc:1514] Processing DeleteTablet for tablet a6d7adb55412482cbe43a89cc2b97f18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:37 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:43214
I20260808 16:56:37.798125 10619 raft_consensus.cc:2229] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:56:37.798161 10616 tablet_replica.cc:330] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15: stopping tablet replica
I20260808 16:56:37.798192 10619 raft_consensus.cc:2258] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:37.798215 10616 raft_consensus.cc:2229] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:56:37.798252 10616 pending_rounds.cc:62] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15: Trying to abort 2 pending ops.
I20260808 16:56:37.798262 10618 tablet_replica.cc:330] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749: stopping tablet replica
I20260808 16:56:37.798310 10618 raft_consensus.cc:2229] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:56:37.798269 10616 pending_rounds.cc:68] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316308778127958016 op_type: PARTICIPANT_OP participant_request { tablet_id: "a6d7adb55412482cbe43a89cc2b97f18" op { type: BEGIN_TXN txn_id: 0 } }
I20260808 16:56:37.798367 10618 raft_consensus.cc:2258] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 16:56:37.798393 10660 consensus_queue.cc:1159] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9f2546443b7c429f82f4931b116f6e15" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260808 16:56:37.798393 10616 pending_rounds.cc:68] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316308777471922176 op_type: NO_OP noop_request { }
I20260808 16:56:37.798596 10616 raft_consensus.cc:2866] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260808 16:56:37.798635 10616 raft_consensus.cc:2258] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:37.798723 10618 ts_tablet_manager.cc:1905] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:37.798964 10616 ts_tablet_manager.cc:1905] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:37.800474 10619 ts_tablet_manager.cc:1905] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:37.800544 10616 ts_tablet_manager.cc:1918] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:56:37.800546 10618 ts_tablet_manager.cc:1918] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:56:37.800606 10618 log.cc:1196] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/a6d7adb55412482cbe43a89cc2b97f18
I20260808 16:56:37.800604 10616 log.cc:1196] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/a6d7adb55412482cbe43a89cc2b97f18
I20260808 16:56:37.800827 10618 ts_tablet_manager.cc:1939] T a6d7adb55412482cbe43a89cc2b97f18 P d67dc6fa7aef4b9aa80345cdcf2a4749: Deleting consensus metadata
I20260808 16:56:37.800837 10616 ts_tablet_manager.cc:1939] T a6d7adb55412482cbe43a89cc2b97f18 P 9f2546443b7c429f82f4931b116f6e15: Deleting consensus metadata
I20260808 16:56:37.801108 9835 catalog_manager.cc:4841] TS d67dc6fa7aef4b9aa80345cdcf2a4749 (127.8.14.66:42205): tablet a6d7adb55412482cbe43a89cc2b97f18 (table test-table [id=b55ca5277f824d45957aab4606d90e9b]) successfully deleted
I20260808 16:56:37.801295 9836 catalog_manager.cc:4841] TS 9f2546443b7c429f82f4931b116f6e15 (127.8.14.67:34243): tablet a6d7adb55412482cbe43a89cc2b97f18 (table test-table [id=b55ca5277f824d45957aab4606d90e9b]) successfully deleted
I20260808 16:56:37.801936 10619 ts_tablet_manager.cc:1918] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:56:37.802019 10619 log.cc:1196] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786208190438830-8249-0/minicluster-data/ts-0/wal/wals/a6d7adb55412482cbe43a89cc2b97f18
I20260808 16:56:37.802240 10619 ts_tablet_manager.cc:1939] T a6d7adb55412482cbe43a89cc2b97f18 P d3985510e5db45529e0de98fa65a1853: Deleting consensus metadata
I20260808 16:56:37.802568 9834 catalog_manager.cc:4841] TS d3985510e5db45529e0de98fa65a1853 (127.8.14.65:40855): tablet a6d7adb55412482cbe43a89cc2b97f18 (table test-table [id=b55ca5277f824d45957aab4606d90e9b]) successfully deleted
Aug 08 16:56:37 dist-test-slave-1nl6 krb5kdc[9793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208196, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.14.65@KRBTEST.COM
I20260808 16:56:37.811056 10576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:37.802700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49080 (local address 127.8.14.65:40855)
0808 16:56:37.802742 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:37.802744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:37.802965 (+ 221us) server_negotiation.cc:407] Connection header received
0808 16:56:37.803039 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:37.803040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:37.803051 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:37.803106 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:37.803672 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.804140 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.804862 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.805067 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.805146 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:37.808136 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:37.808145 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:37.808145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:37.808153 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:37.809771 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:37.810158 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:37.810162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:37.810162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:37.810250 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:37.810465 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:37.810466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:37.810467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:37.810611 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:37.810767 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:37.810835 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 16:56:37.810958 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 16:56:37.811277 10131 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49080
I20260808 16:56:37.818399 10634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:37.812296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59308 (local address 127.8.14.126:40937)
0808 16:56:37.812355 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:37.812357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:37.812409 (+ 52us) server_negotiation.cc:407] Connection header received
0808 16:56:37.812488 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:37.812489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:37.812499 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:37.812554 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:37.813064 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.813552 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.814248 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:37.814462 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:37.814540 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:37.815678 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:37.815680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:37.815681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:37.815688 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:37.817118 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:37.817483 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:37.817484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:37.817484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:37.817541 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:37.817752 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:37.817753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:37.817753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:37.817906 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:37.818033 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:37.818144 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 16:56:37.818230 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 16:56:37.818604 9858 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59308
I20260808 16:56:37.819257 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 9953
I20260808 16:56:37.834020 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 10151
I20260808 16:56:37.843099 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 10357
I20260808 16:56:37.855813 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 9808
2026-08-08T16:56:37Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2923 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:37 dist-test-slave-1nl6 krb5kdc[10670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:37 dist-test-slave-1nl6 krb5kdc[10670](info): set up 2 sockets
Aug 08 16:56:37 dist-test-slave-1nl6 krb5kdc[10670](info): commencing operation
krb5kdc: starting...
W20260808 16:56:39.090242 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:56:39 dist-test-slave-1nl6 krb5kdc[10670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:56:39.211423 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:44293
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:38817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:44293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:39.293272 10686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:39.293512 10686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:39.293552 10686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:39.294674 10686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:39.294755 10686 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 16:56:39.294788 10686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:39.294818 10686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:39.294838 10686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:39.294860 10686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:39.296324 10686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:38817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:44293
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:44293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:39.296607 10686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:39.296837 10686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:39.301716 10697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:39.301697 10694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:39.301689 10695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:39.301790 10686 server_base.cc:1034] running on GCE node
I20260808 16:56:39.302095 10686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:39.302342 10686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:39.303481 10686 hybrid_clock.cc:648] HybridClock initialized: now 1786208199303471 us; error 27 us; skew 500 ppm
Aug 08 16:56:39 dist-test-slave-1nl6 krb5kdc[10670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208199, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:39.305886 10686 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:56:39.306303 10686 webserver.cc:460] Webserver started at http://127.8.14.126:39895/ using document root <none> and password file <none>
I20260808 16:56:39.306529 10686 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:39.306596 10686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:39.306739 10686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:39.307756 10686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "fed7652b01b34b36a37a97faa7d278e3"
format_stamp: "Formatted at 2026-08-08 16:56:39 on dist-test-slave-1nl6"
server_key: "ea26e88d401d2f53eae35994536e3e94"
server_key_iv: "3204bc6bc91d46e8a51054fab95b26a9"
server_key_version: "encryptionkey@0"
I20260808 16:56:39.308137 10686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "fed7652b01b34b36a37a97faa7d278e3"
format_stamp: "Formatted at 2026-08-08 16:56:39 on dist-test-slave-1nl6"
server_key: "ea26e88d401d2f53eae35994536e3e94"
server_key_iv: "3204bc6bc91d46e8a51054fab95b26a9"
server_key_version: "encryptionkey@0"
I20260808 16:56:39.309515 10686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:56:39.310487 10710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:39.310688 10686 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:39.310830 10686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "fed7652b01b34b36a37a97faa7d278e3"
format_stamp: "Formatted at 2026-08-08 16:56:39 on dist-test-slave-1nl6"
server_key: "ea26e88d401d2f53eae35994536e3e94"
server_key_iv: "3204bc6bc91d46e8a51054fab95b26a9"
server_key_version: "encryptionkey@0"
I20260808 16:56:39.310971 10686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:39.322690 10686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:39.324546 10686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:39.324656 10686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:39.329239 10686 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:44293
I20260808 16:56:39.329340 10803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:44293 every 8 connection(s)
I20260808 16:56:39.329687 10686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:39.330386 10804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:39.332721 10804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3: Bootstrap starting.
I20260808 16:56:39.333392 10804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:39.333755 10804 log.cc:824] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:39.334643 10804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3: No bootstrap required, opened a new log
I20260808 16:56:39.336130 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 10686
I20260808 16:56:39.336239 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:56:39.336401 8249 external_mini_cluster.cc:1428] Setting key c00cc2a76a370579c0c973be794414be
I20260808 16:56:39.337224 10804 raft_consensus.cc:348] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed7652b01b34b36a37a97faa7d278e3" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44293 } }
I20260808 16:56:39.337399 10804 raft_consensus.cc:374] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:39.337436 10804 raft_consensus.cc:729] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fed7652b01b34b36a37a97faa7d278e3, State: Initialized, Role: FOLLOWER
I20260808 16:56:39.337568 10804 consensus_queue.cc:260] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed7652b01b34b36a37a97faa7d278e3" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44293 } }
I20260808 16:56:39.337667 10804 raft_consensus.cc:388] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:39.337702 10804 raft_consensus.cc:482] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:39.337754 10804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:39.338511 10804 raft_consensus.cc:504] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed7652b01b34b36a37a97faa7d278e3" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44293 } }
I20260808 16:56:39.338629 10804 leader_election.cc:302] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fed7652b01b34b36a37a97faa7d278e3; no voters:
I20260808 16:56:39.338802 10804 leader_election.cc:288] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:39.338853 10813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:39.339095 10804 sys_catalog.cc:564] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:39.339116 10813 raft_consensus.cc:686] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [term 1 LEADER]: Becoming Leader. State: Replica: fed7652b01b34b36a37a97faa7d278e3, State: Running, Role: LEADER
I20260808 16:56:39.339350 10813 consensus_queue.cc:237] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed7652b01b34b36a37a97faa7d278e3" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44293 } }
I20260808 16:56:39.339812 10815 sys_catalog.cc:455] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fed7652b01b34b36a37a97faa7d278e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed7652b01b34b36a37a97faa7d278e3" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44293 } } }
I20260808 16:56:39.339810 10813 sys_catalog.cc:455] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fed7652b01b34b36a37a97faa7d278e3. Latest consensus state: current_term: 1 leader_uuid: "fed7652b01b34b36a37a97faa7d278e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fed7652b01b34b36a37a97faa7d278e3" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44293 } } }
I20260808 16:56:39.339960 10815 sys_catalog.cc:458] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:39.340205 10813 sys_catalog.cc:458] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:39.340230 10827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:39.340786 10827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 16:56:39 dist-test-slave-1nl6 krb5kdc[10670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:39.344525 10827 catalog_manager.cc:1347] Generated new cluster ID: 47953623c5274aa68e220c2a87c4e342
I20260808 16:56:39.344599 10827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:39.348440 10811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.336961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47606 (local address 127.8.14.126:44293)
0808 16:56:39.337191 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:39.337201 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:39.337230 (+ 29us) server_negotiation.cc:407] Connection header received
0808 16:56:39.337351 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:39.337359 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:39.337413 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:39.338379 (+ 966us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:39.338482 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.339976 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.341146 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.341735 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.341866 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.344511 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:39.344515 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:39.344515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:39.344527 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:39.346567 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:39.346995 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:39.346998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:39.346998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:39.347119 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:39.347302 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:39.347303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:39.347303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:39.347581 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:39.347710 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:39.348120 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 16:56:39.348252 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":103,"threads_started":1}
I20260808 16:56:39.350099 10827 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:39.350594 10827 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:39.352959 10827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3: Generated new TSK 0
I20260808 16:56:39.353123 10827 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 16:56:39.384114 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:44293
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:38817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:39.486480 10846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:39.486712 10846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:39.486754 10846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:39.487946 10846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:39.488018 10846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:39.488054 10846 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 16:56:39.488080 10846 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:56:39.488116 10846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:56:39.489620 10846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:38817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.14.126:44293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:39.489879 10846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:39.490108 10846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:39.495241 10859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:39.495344 10846 server_base.cc:1034] running on GCE node
W20260808 16:56:39.495318 10852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:39.495278 10855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:39.495644 10846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:39.495899 10846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:39.497040 10846 hybrid_clock.cc:648] HybridClock initialized: now 1786208199497027 us; error 27 us; skew 500 ppm
Aug 08 16:56:39 dist-test-slave-1nl6 krb5kdc[10670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208199, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:39.499590 10846 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:56:39.500037 10846 webserver.cc:460] Webserver started at http://127.8.14.65:42219/ using document root <none> and password file <none>
I20260808 16:56:39.500236 10846 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:39.500276 10846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:39.500361 10846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:39.501293 10846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "b451aecf307d4e7eb08540ba338eb1d3"
format_stamp: "Formatted at 2026-08-08 16:56:39 on dist-test-slave-1nl6"
server_key: "d32341760b8988e77378c7bc1a1d9ac6"
server_key_iv: "40d9caf56b19aaf18233ef61cfbf1cc4"
server_key_version: "encryptionkey@0"
I20260808 16:56:39.501593 10846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "b451aecf307d4e7eb08540ba338eb1d3"
format_stamp: "Formatted at 2026-08-08 16:56:39 on dist-test-slave-1nl6"
server_key: "d32341760b8988e77378c7bc1a1d9ac6"
server_key_iv: "40d9caf56b19aaf18233ef61cfbf1cc4"
server_key_version: "encryptionkey@0"
I20260808 16:56:39.502843 10846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 16:56:39.503595 10867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:39.503772 10846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:39.503875 10846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "b451aecf307d4e7eb08540ba338eb1d3"
format_stamp: "Formatted at 2026-08-08 16:56:39 on dist-test-slave-1nl6"
server_key: "d32341760b8988e77378c7bc1a1d9ac6"
server_key_iv: "40d9caf56b19aaf18233ef61cfbf1cc4"
server_key_version: "encryptionkey@0"
I20260808 16:56:39.503958 10846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:39.528545 10846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:39.531489 10846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:39.531600 10846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:39.532133 10846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:39.532182 10846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:39.532204 10846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:39.532217 10846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 16:56:39 dist-test-slave-1nl6 krb5kdc[10670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208199, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:39.539083 10846 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:38483
I20260808 16:56:39.539518 10846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:56:39.540126 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 10846
I20260808 16:56:39.540223 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:56:39.540426 8249 external_mini_cluster.cc:1428] Setting key f9096b5c21a3a2cd5952ed963037b0ec
I20260808 16:56:39.539995 11023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:38483 every 8 connection(s)
I20260808 16:56:39.542419 10811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.532521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:38393 (local address 127.8.14.126:44293)
0808 16:56:39.532579 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:39.532581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:39.532802 (+ 221us) server_negotiation.cc:407] Connection header received
0808 16:56:39.533277 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:39.533278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:39.533294 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:39.533387 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:39.533950 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.534765 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.536374 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.536754 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.536847 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.539141 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:39.539143 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:39.539143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:39.539151 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:39.540931 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:39.541412 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:39.541414 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:39.541415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:39.541476 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:39.541743 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:39.541744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:39.541744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:39.541869 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:39.541946 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:39.542237 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 16:56:39.542329 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 16:56:39.542596 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.532501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44293 (local address 127.8.14.65:38393)
0808 16:56:39.532667 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:39.532707 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:39.533181 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:39.533436 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:39.533439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:39.533657 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:39.533836 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.533840 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.534861 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.534862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.536270 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.536274 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.536877 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.536878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.537008 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.537516 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:39.537524 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:39.539014 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:39.541034 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:39.541041 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:39.541044 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:39.541310 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:39.541552 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:39.541553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:39.541553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:39.541668 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:39.541949 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:39.541957 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:39.542094 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 16:56:39.542199 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 16:56:39.542363 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":100,"threads_started":1}
I20260808 16:56:39.543197 11024 heartbeater.cc:347] Connected to a master server at 127.8.14.126:44293
I20260808 16:56:39.543301 11024 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:39.543488 11024 heartbeater.cc:511] Master 127.8.14.126:44293 requested a full tablet report, sending...
I20260808 16:56:39.544023 10734 ts_manager.cc:194] Registered new tserver with Master: b451aecf307d4e7eb08540ba338eb1d3 (127.8.14.65:38483)
I20260808 16:56:39.544718 10734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:38393
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
I20260808 16:56:39.550702 10811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.544324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:58753 (local address 127.8.14.126:44293)
0808 16:56:39.544371 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:39.544374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:39.544510 (+ 136us) server_negotiation.cc:407] Connection header received
0808 16:56:39.544599 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:39.544600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:39.544610 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:39.544638 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:39.545086 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.545894 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.547051 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.547229 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.547286 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.548260 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:39.548261 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:39.548262 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:39.548267 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:39.549687 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:39.550094 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:39.550095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:39.550097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:39.550160 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:39.550354 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:39.550355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:39.550355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:39.550441 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:39.550485 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:39.550588 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 16:56:39.550645 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:56:39.550709 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.544259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44293 (local address 127.8.14.65:58753)
0808 16:56:39.544452 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:39.544462 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:39.544535 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:39.544697 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:39.544698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:39.544919 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:39.544984 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.544986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.546026 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.546026 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.546958 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.546961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.547314 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.547316 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.547403 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.547783 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:39.547789 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:39.548165 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:39.549780 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:39.549781 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:39.549781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:39.549989 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:39.550210 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:39.550211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:39.550212 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:39.550282 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:39.550485 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:39.550486 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:39.550523 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 16:56:39.550573 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 16:56:39.550648 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":85,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 16:56:39.567981 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:44293
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:38817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:39.648209 11039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:39.648391 11039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:39.648414 11039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:39.649600 11039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:39.649644 11039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:39.649684 11039 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 16:56:39.649729 11039 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:56:39.649765 11039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:56:39.651212 11039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:38817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.14.126:44293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:39.651468 11039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:39.651705 11039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:39.656770 11047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:39.656793 11046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:39.656793 11050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:39.656908 11039 server_base.cc:1034] running on GCE node
I20260808 16:56:39.657167 11039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:39.657388 11039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:39.658528 11039 hybrid_clock.cc:648] HybridClock initialized: now 1786208199658494 us; error 63 us; skew 500 ppm
Aug 08 16:56:39 dist-test-slave-1nl6 krb5kdc[10670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208199, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:39.661001 11039 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:56:39.661617 11039 webserver.cc:460] Webserver started at http://127.8.14.66:39651/ using document root <none> and password file <none>
I20260808 16:56:39.661900 11039 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:39.661978 11039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:39.662143 11039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:39.663275 11039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "5c498df9fc364f6b8876f6c89e20c3b4"
format_stamp: "Formatted at 2026-08-08 16:56:39 on dist-test-slave-1nl6"
server_key: "2a7ba85fde2f280a21bd6d3cfe5f1fc5"
server_key_iv: "7c63efc333acbd58a02b61366289df4a"
server_key_version: "encryptionkey@0"
I20260808 16:56:39.663645 11039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "5c498df9fc364f6b8876f6c89e20c3b4"
format_stamp: "Formatted at 2026-08-08 16:56:39 on dist-test-slave-1nl6"
server_key: "2a7ba85fde2f280a21bd6d3cfe5f1fc5"
server_key_iv: "7c63efc333acbd58a02b61366289df4a"
server_key_version: "encryptionkey@0"
I20260808 16:56:39.664968 11039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:56:39.665649 11060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:39.665836 11039 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:39.665946 11039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "5c498df9fc364f6b8876f6c89e20c3b4"
format_stamp: "Formatted at 2026-08-08 16:56:39 on dist-test-slave-1nl6"
server_key: "2a7ba85fde2f280a21bd6d3cfe5f1fc5"
server_key_iv: "7c63efc333acbd58a02b61366289df4a"
server_key_version: "encryptionkey@0"
I20260808 16:56:39.666033 11039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:39.675280 11039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:39.677078 11039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:39.677352 11039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:39.677914 11039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:39.677981 11039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:39.678052 11039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:39.678085 11039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 16:56:39 dist-test-slave-1nl6 krb5kdc[10670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208199, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:39.685245 11039 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:38605
I20260808 16:56:39.685375 11226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:38605 every 8 connection(s)
I20260808 16:56:39.685693 11039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:56:39.687853 10811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.678261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:47579 (local address 127.8.14.126:44293)
0808 16:56:39.678307 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:39.678309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:39.678629 (+ 320us) server_negotiation.cc:407] Connection header received
0808 16:56:39.679316 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:39.679318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:39.679328 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:39.679370 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:39.680224 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.680699 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.681674 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.681885 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.681969 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.684781 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:39.684783 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:39.684783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:39.684790 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:39.686419 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:39.686857 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:39.686859 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:39.686859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:39.686939 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:39.687195 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:39.687196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:39.687197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:39.687324 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:39.687404 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:39.687684 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 16:56:39.687765 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:56:39.688014 11072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.678251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44293 (local address 127.8.14.66:47579)
0808 16:56:39.678470 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:39.678512 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:39.679210 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:39.679443 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:39.679448 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:39.679837 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:39.680123 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.680129 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.680814 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.680815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.681578 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.681580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.681976 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.681977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.682150 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.682709 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:39.682717 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:39.684674 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:39.686534 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:39.686540 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:39.686542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:39.686762 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:39.687003 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:39.687003 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:39.687004 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:39.687111 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:39.687395 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:39.687402 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:39.687540 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 16:56:39.687644 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 16:56:39.687801 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":106,"threads_started":1}
I20260808 16:56:39.688556 11227 heartbeater.cc:347] Connected to a master server at 127.8.14.126:44293
I20260808 16:56:39.688639 11227 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:39.688814 11227 heartbeater.cc:511] Master 127.8.14.126:44293 requested a full tablet report, sending...
I20260808 16:56:39.689177 10732 ts_manager.cc:194] Registered new tserver with Master: 5c498df9fc364f6b8876f6c89e20c3b4 (127.8.14.66:38605)
I20260808 16:56:39.689670 10732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:47579
I20260808 16:56:39.692631 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 11039
I20260808 16:56:39.692740 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:56:39.692929 8249 external_mini_cluster.cc:1428] Setting key 00518275f40502200b974716d47535ef
I20260808 16:56:39.695451 10811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.689596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:42937 (local address 127.8.14.126:44293)
0808 16:56:39.689629 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:39.689630 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:39.689788 (+ 158us) server_negotiation.cc:407] Connection header received
0808 16:56:39.689811 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:39.689812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:39.689822 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:39.689876 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:39.690231 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.690735 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.691514 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.691810 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.691867 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.692764 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:39.692766 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:39.692766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:39.692773 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:39.694227 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:39.694572 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:39.694573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:39.694574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:39.694629 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:39.694845 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:39.694846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:39.694846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:39.695045 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:39.695130 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:39.695286 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 16:56:39.695363 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:56:39.695464 11234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.689527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44293 (local address 127.8.14.66:42937)
0808 16:56:39.689669 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:39.689679 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:39.689737 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:39.689897 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:39.689897 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:39.690094 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:39.690148 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.690149 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.690822 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.690822 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.691444 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.691446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.691881 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.691881 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.691936 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.692376 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:39.692381 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:39.692644 (+ 263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:39.694322 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:39.694323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:39.694324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:39.694494 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:39.694709 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:39.694710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:39.694710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:39.694770 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:39.695124 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:39.695125 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:39.695171 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 16:56:39.695261 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 16:56:39.695364 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 16:56:39.719036 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:44293
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:38817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:39.800925 11239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:39.801111 11239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:39.801133 11239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:39.802232 11239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:39.802273 11239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:39.802295 11239 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 16:56:39.802307 11239 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:56:39.802323 11239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:56:39.803894 11239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:38817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.14.126:44293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:39.804140 11239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:39.804385 11239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:39.809439 11251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:39.809456 11247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:39.809510 11239 server_base.cc:1034] running on GCE node
W20260808 16:56:39.809440 11248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:39.809893 11239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:39.810178 11239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:39.811367 11239 hybrid_clock.cc:648] HybridClock initialized: now 1786208199811338 us; error 41 us; skew 500 ppm
Aug 08 16:56:39 dist-test-slave-1nl6 krb5kdc[10670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208199, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:39.813959 11239 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:56:39.814457 11239 webserver.cc:460] Webserver started at http://127.8.14.67:41145/ using document root <none> and password file <none>
I20260808 16:56:39.814688 11239 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:39.814759 11239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:39.814958 11239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:39.816004 11239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "e90f814c78a441ca8417c9e052e726be"
format_stamp: "Formatted at 2026-08-08 16:56:39 on dist-test-slave-1nl6"
server_key: "aa3f7119548b3ec0962149eea6d4f146"
server_key_iv: "aa9d349415d90a33347bf61d17f5850f"
server_key_version: "encryptionkey@0"
I20260808 16:56:39.816387 11239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "e90f814c78a441ca8417c9e052e726be"
format_stamp: "Formatted at 2026-08-08 16:56:39 on dist-test-slave-1nl6"
server_key: "aa3f7119548b3ec0962149eea6d4f146"
server_key_iv: "aa9d349415d90a33347bf61d17f5850f"
server_key_version: "encryptionkey@0"
I20260808 16:56:39.817727 11239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:56:39.818475 11265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:39.818670 11239 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:56:39.818799 11239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "e90f814c78a441ca8417c9e052e726be"
format_stamp: "Formatted at 2026-08-08 16:56:39 on dist-test-slave-1nl6"
server_key: "aa3f7119548b3ec0962149eea6d4f146"
server_key_iv: "aa9d349415d90a33347bf61d17f5850f"
server_key_version: "encryptionkey@0"
I20260808 16:56:39.818934 11239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:39.831748 11239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:39.834151 11239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:39.834336 11239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:39.834843 11239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:39.834940 11239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:39.835002 11239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:39.835033 11239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:39.842171 11239 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:46385
I20260808 16:56:39.842229 11423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:46385 every 8 connection(s)
Aug 08 16:56:39 dist-test-slave-1nl6 krb5kdc[10670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208199, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:39.842615 11239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 16:56:39.843729 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 11239
I20260808 16:56:39.843845 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:56:39.844075 8249 external_mini_cluster.cc:1428] Setting key 80155b337ea114eabc0b63c48cfedb6c
I20260808 16:56:39.846153 10811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.835387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:53283 (local address 127.8.14.126:44293)
0808 16:56:39.835460 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:39.835463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:39.835648 (+ 185us) server_negotiation.cc:407] Connection header received
0808 16:56:39.836106 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:39.836107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:39.836129 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:39.836214 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:39.837083 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.837919 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.839483 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.839808 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.839896 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.843035 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:39.843037 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:39.843038 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:39.843045 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:39.844788 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:39.845278 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:39.845284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:39.845284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:39.845341 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:39.845564 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:39.845564 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:39.845565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:39.845664 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:39.845713 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:39.846007 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 16:56:39.846090 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 16:56:39.846354 11277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.835372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44293 (local address 127.8.14.67:53283)
0808 16:56:39.835520 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:39.835560 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:39.836012 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:39.836262 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:39.836265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:39.836670 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:39.836966 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.836972 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.838053 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.838054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.839405 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.839409 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.839976 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.839977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.840092 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.841165 (+ 1073us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:39.841176 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:39.842913 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:39.844860 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:39.844865 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:39.844867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:39.845203 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:39.845406 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:39.845406 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:39.845407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:39.845504 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:39.845705 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:39.845708 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:39.845846 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 16:56:39.845956 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 16:56:39.846108 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260808 16:56:39.846870 11428 heartbeater.cc:347] Connected to a master server at 127.8.14.126:44293
I20260808 16:56:39.846976 11428 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:39.847157 11428 heartbeater.cc:511] Master 127.8.14.126:44293 requested a full tablet report, sending...
I20260808 16:56:39.847532 10732 ts_manager.cc:194] Registered new tserver with Master: e90f814c78a441ca8417c9e052e726be (127.8.14.67:46385)
I20260808 16:56:39.848019 10732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:53283
I20260808 16:56:39.853070 10811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.847979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:35811 (local address 127.8.14.126:44293)
0808 16:56:39.848023 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:39.848025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:39.848171 (+ 146us) server_negotiation.cc:407] Connection header received
0808 16:56:39.848277 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:39.848278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:39.848318 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:39.848348 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:39.848710 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.849202 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.849866 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.850042 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.850091 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.850855 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:39.850856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:39.850856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:39.850862 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:39.852003 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:39.852340 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:39.852341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:39.852341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:39.852392 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:39.852570 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:39.852570 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:39.852571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:39.852673 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:39.852723 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:39.852923 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 16:56:39.852993 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:56:39.853325 11440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.847909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44293 (local address 127.8.14.67:35811)
0808 16:56:39.848078 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:39.848096 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:39.848200 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:39.848410 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:39.848410 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:39.848567 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:39.848625 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.848626 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.849315 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.849316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.849791 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.849792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.850111 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.850112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.850165 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.850498 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:39.850502 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:39.850772 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:39.852081 (+ 1309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:39.852081 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:39.852082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:39.852258 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:39.852455 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:39.852456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:39.852456 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:39.852510 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:39.852730 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:39.852731 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:39.852794 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 16:56:39.852877 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 16:56:39.852979 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":81,"threads_started":1}
I20260808 16:56:39.855844 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 16:56:39 dist-test-slave-1nl6 krb5kdc[10670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 16:56:39 dist-test-slave-1nl6 krb5kdc[10670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:39.878965 10811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.872246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47622 (local address 127.8.14.126:44293)
0808 16:56:39.872306 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:39.872308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:39.872331 (+ 23us) server_negotiation.cc:407] Connection header received
0808 16:56:39.872388 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:39.872389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:39.872396 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:39.872444 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:39.872983 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.873431 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.874102 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.874271 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.874320 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.876564 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:39.876566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:39.876566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:39.876574 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:39.877748 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:39.878133 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:39.878134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:39.878134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:39.878193 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:39.878417 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:39.878418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:39.878418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:39.878522 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:39.878629 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:39.878728 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 16:56:39.878815 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 16:56:39.880597 10732 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:56:39.882088 10732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:56:39.888481 11460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.884340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:38605 (local address 127.0.0.1:47560)
0808 16:56:39.884498 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:39.884538 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:39.884604 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:39.884957 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:39.884960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:39.884962 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:39.885112 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.885115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.886031 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.886032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.887269 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.887272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.888183 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.888185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.888250 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.888254 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:56:39.888302 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 16:56:39.888371 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":102,"threads_started":1}
I20260808 16:56:39.888492 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.884574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43338 (local address 127.8.14.67:46385)
0808 16:56:39.884756 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:39.884761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:39.884864 (+ 103us) server_negotiation.cc:407] Connection header received
0808 16:56:39.884910 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:39.884912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:39.884927 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:39.884996 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:39.885203 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.886286 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.887542 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.888025 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.888120 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.888182 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:39.888363 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 16:56:39.888399 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":83,"threads_started":1}
I20260808 16:56:39.888662 11461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.884340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:38483 (local address 127.0.0.1:44594)
0808 16:56:39.884772 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:39.884783 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:39.884851 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:39.885234 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:39.885235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:39.885237 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:39.885268 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.885269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.886505 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.886507 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.887489 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.887492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.888454 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.888455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.888512 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.888516 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:56:39.888566 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 16:56:39.888599 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":3,"thread_start_us":45,"threads_started":1}
I20260808 16:56:39.888724 11464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.884593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47560 (local address 127.8.14.66:38605)
0808 16:56:39.884768 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:39.884771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:39.884781 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:56:39.884825 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:39.884851 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:39.884868 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:39.884956 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:39.885182 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.885935 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.887361 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.888079 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.888440 (+ 361us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.888532 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:39.888559 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 16:56:39.888597 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260808 16:56:39.888756 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.884353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:46385 (local address 127.0.0.1:43338)
0808 16:56:39.884771 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:39.884783 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:39.884851 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:39.885060 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:39.885061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:39.885064 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:39.885121 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.885124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.886386 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.886387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.887450 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:39.887453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.888124 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.888125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:39.888217 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.888231 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 16:56:39.888344 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 16:56:39.888390 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":95,"threads_started":1}
I20260808 16:56:39.888999 11462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:39.884523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44594 (local address 127.8.14.65:38483)
0808 16:56:39.884717 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:39.884722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:39.884912 (+ 190us) server_negotiation.cc:407] Connection header received
0808 16:56:39.884960 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:39.884962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:39.884979 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:39.885058 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:39.885365 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.886415 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.887583 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:39.888283 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:39.888361 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:39.888419 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:39.888865 (+ 446us) server_negotiation.cc:299] Negotiation successful
0808 16:56:39.888911 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":84,"threads_started":1}
I20260808 16:56:39.889483 11133 tablet_service.cc:1467] Processing CreateTablet for tablet dae3b496ea2544579fb42dcbc756a772 (DEFAULT_TABLE table=test-table [id=008ca597472b470195afba2b58e13cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:39.889902 11338 tablet_service.cc:1467] Processing CreateTablet for tablet dae3b496ea2544579fb42dcbc756a772 (DEFAULT_TABLE table=test-table [id=008ca597472b470195afba2b58e13cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:39.889997 11133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dae3b496ea2544579fb42dcbc756a772. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:39.890177 11338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dae3b496ea2544579fb42dcbc756a772. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:39.891537 10935 tablet_service.cc:1467] Processing CreateTablet for tablet dae3b496ea2544579fb42dcbc756a772 (DEFAULT_TABLE table=test-table [id=008ca597472b470195afba2b58e13cec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:39.892060 10935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dae3b496ea2544579fb42dcbc756a772. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:39.892954 11465 tablet_bootstrap.cc:492] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be: Bootstrap starting.
I20260808 16:56:39.893699 11468 tablet_bootstrap.cc:492] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4: Bootstrap starting.
I20260808 16:56:39.893754 11465 tablet_bootstrap.cc:654] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:39.894117 11465 log.cc:824] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:39.894351 11468 tablet_bootstrap.cc:654] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:39.894657 11468 log.cc:824] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:39.894956 11465 tablet_bootstrap.cc:492] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be: No bootstrap required, opened a new log
I20260808 16:56:39.895017 11465 ts_tablet_manager.cc:1397] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:56:39.895522 11468 tablet_bootstrap.cc:492] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4: No bootstrap required, opened a new log
I20260808 16:56:39.895577 11468 ts_tablet_manager.cc:1397] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:56:39.895624 11471 tablet_bootstrap.cc:492] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3: Bootstrap starting.
I20260808 16:56:39.896268 11471 tablet_bootstrap.cc:654] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:39.896553 11471 log.cc:824] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:39.896992 11465 raft_consensus.cc:348] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:39.897249 11465 raft_consensus.cc:374] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:39.897324 11471 tablet_bootstrap.cc:492] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3: No bootstrap required, opened a new log
I20260808 16:56:39.897347 11465 raft_consensus.cc:729] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90f814c78a441ca8417c9e052e726be, State: Initialized, Role: FOLLOWER
I20260808 16:56:39.897435 11471 ts_tablet_manager.cc:1397] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:56:39.897476 11468 raft_consensus.cc:348] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:39.897554 11465 consensus_queue.cc:260] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:39.897673 11468 raft_consensus.cc:374] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:39.897758 11468 raft_consensus.cc:729] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c498df9fc364f6b8876f6c89e20c3b4, State: Initialized, Role: FOLLOWER
I20260808 16:56:39.897835 11465 ts_tablet_manager.cc:1428] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:56:39.897897 11468 consensus_queue.cc:260] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:39.897951 11428 heartbeater.cc:503] Master 127.8.14.126:44293 was elected leader, sending a full tablet report...
I20260808 16:56:39.898190 11227 heartbeater.cc:503] Master 127.8.14.126:44293 was elected leader, sending a full tablet report...
I20260808 16:56:39.898463 11468 ts_tablet_manager.cc:1428] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:56:39.899451 11471 raft_consensus.cc:348] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:39.899570 11471 raft_consensus.cc:374] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:39.899591 11471 raft_consensus.cc:729] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b451aecf307d4e7eb08540ba338eb1d3, State: Initialized, Role: FOLLOWER
I20260808 16:56:39.899729 11471 consensus_queue.cc:260] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:39.899991 11471 ts_tablet_manager.cc:1428] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 16:56:39.900058 11024 heartbeater.cc:503] Master 127.8.14.126:44293 was elected leader, sending a full tablet report...
W20260808 16:56:39.936286 11228 tablet.cc:2369] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:56:40.025441 11475 raft_consensus.cc:482] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:56:40.025631 11475 raft_consensus.cc:504] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.026041 11475 leader_election.cc:288] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c498df9fc364f6b8876f6c89e20c3b4 (127.8.14.66:38605), e90f814c78a441ca8417c9e052e726be (127.8.14.67:46385)
I20260808 16:56:40.029039 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.026381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:43361 (local address 127.8.14.67:46385)
0808 16:56:40.026430 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:40.026432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:40.026442 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:56:40.026476 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:40.026477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:40.026489 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:40.026548 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:40.026740 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.027316 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.028302 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.028651 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.028705 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.028714 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:40.028912 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 16:56:40.028961 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:56:40.029043 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.026248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:46385 (local address 127.8.14.65:43361)
0808 16:56:40.026316 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:40.026330 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:40.026396 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:40.026565 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:40.026566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:40.026568 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:40.026658 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.026661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.027403 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.027404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.028199 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.028201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.028737 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.028738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.028827 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.028831 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:56:40.028884 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 16:56:40.028915 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":18}
I20260808 16:56:40.029088 11464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.026357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:55383 (local address 127.8.14.66:38605)
0808 16:56:40.026404 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:40.026406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:40.026565 (+ 159us) server_negotiation.cc:407] Connection header received
0808 16:56:40.026595 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:40.026596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:40.026608 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:40.026663 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:40.026869 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.027412 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.028342 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.028705 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.028779 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.028788 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:40.028971 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 16:56:40.029015 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 16:56:40.029146 11476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.026248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:38605 (local address 127.8.14.65:55383)
0808 16:56:40.026443 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:40.026454 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:40.026519 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:40.026681 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:40.026682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:40.026683 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:40.026793 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.026795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.027491 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.027492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.028210 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.028212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.028803 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.028804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.028863 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.028887 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 16:56:40.028947 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 16:56:40.028983 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":57,"thread_start_us":85,"threads_started":1}
I20260808 16:56:40.029462 11365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dae3b496ea2544579fb42dcbc756a772" candidate_uuid: "b451aecf307d4e7eb08540ba338eb1d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90f814c78a441ca8417c9e052e726be" is_pre_election: true
I20260808 16:56:40.029665 11365 raft_consensus.cc:2454] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b451aecf307d4e7eb08540ba338eb1d3 in term 0.
I20260808 16:56:40.029667 11164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dae3b496ea2544579fb42dcbc756a772" candidate_uuid: "b451aecf307d4e7eb08540ba338eb1d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" is_pre_election: true
I20260808 16:56:40.029829 11164 raft_consensus.cc:2454] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b451aecf307d4e7eb08540ba338eb1d3 in term 0.
I20260808 16:56:40.029971 10868 leader_election.cc:302] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b451aecf307d4e7eb08540ba338eb1d3, e90f814c78a441ca8417c9e052e726be; no voters:
I20260808 16:56:40.030119 11475 raft_consensus.cc:2781] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:56:40.030179 11475 raft_consensus.cc:482] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:56:40.030201 11475 raft_consensus.cc:3037] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:40.030973 11475 raft_consensus.cc:504] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.031129 11475 leader_election.cc:288] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [CANDIDATE]: Term 1 election: Requested vote from peers 5c498df9fc364f6b8876f6c89e20c3b4 (127.8.14.66:38605), e90f814c78a441ca8417c9e052e726be (127.8.14.67:46385)
I20260808 16:56:40.031282 11365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dae3b496ea2544579fb42dcbc756a772" candidate_uuid: "b451aecf307d4e7eb08540ba338eb1d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90f814c78a441ca8417c9e052e726be"
I20260808 16:56:40.031282 11164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dae3b496ea2544579fb42dcbc756a772" candidate_uuid: "b451aecf307d4e7eb08540ba338eb1d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c498df9fc364f6b8876f6c89e20c3b4"
I20260808 16:56:40.031365 11164 raft_consensus.cc:3037] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:40.031365 11365 raft_consensus.cc:3037] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:40.032244 11365 raft_consensus.cc:2454] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b451aecf307d4e7eb08540ba338eb1d3 in term 1.
I20260808 16:56:40.032244 11164 raft_consensus.cc:2454] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b451aecf307d4e7eb08540ba338eb1d3 in term 1.
I20260808 16:56:40.032436 10871 leader_election.cc:302] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c498df9fc364f6b8876f6c89e20c3b4, b451aecf307d4e7eb08540ba338eb1d3; no voters:
I20260808 16:56:40.032531 11475 raft_consensus.cc:2781] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:40.032701 11475 raft_consensus.cc:686] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [term 1 LEADER]: Becoming Leader. State: Replica: b451aecf307d4e7eb08540ba338eb1d3, State: Running, Role: LEADER
I20260808 16:56:40.032832 11475 consensus_queue.cc:237] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.033682 10732 catalog_manager.cc:5495] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to b451aecf307d4e7eb08540ba338eb1d3 (127.8.14.65). New cstate: current_term: 1 leader_uuid: "b451aecf307d4e7eb08540ba338eb1d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } health_report { overall_health: UNKNOWN } } }
W20260808 16:56:40.040500 11027 tablet.cc:2369] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 16:56:40 dist-test-slave-1nl6 krb5kdc[10670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208199, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
W20260808 16:56:40.093338 11433 tablet.cc:2369] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:56:40.094594 11486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.087697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44293 (local address 127.0.0.1:47626)
0808 16:56:40.087899 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:40.087910 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:40.088005 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:40.088161 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:40.088162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:40.088512 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:40.088560 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.088563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.089236 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.089237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.089745 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.089747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.090106 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.090106 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.090161 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.090652 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:40.090657 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:40.092000 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:40.093692 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:40.093699 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:40.093701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:40.093893 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:40.094075 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:40.094076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:40.094076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:40.094132 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:40.094341 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:40.094347 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:40.094388 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 16:56:40.094440 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 16:56:40.094525 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":101,"threads_started":1}
I20260808 16:56:40.094655 10811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.087785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47626 (local address 127.8.14.126:44293)
0808 16:56:40.087889 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:40.087891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:40.088005 (+ 114us) server_negotiation.cc:407] Connection header received
0808 16:56:40.088072 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:40.088073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:40.088082 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:40.088143 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:40.088650 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.089136 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.089841 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.090027 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.090106 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.092100 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:40.092102 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:40.092102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:40.092126 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:40.093604 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:40.093962 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:40.093963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:40.093963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:40.094017 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:40.094186 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:40.094187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:40.094187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:40.094285 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:40.094337 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:40.094473 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 16:56:40.094573 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":72}
I20260808 16:56:40.096637 10732 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:47626:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 16:56:40.096860 10732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:56:40.098333 10935 tablet_service.cc:1467] Processing CreateTablet for tablet 6f4f6dfbb2f04447971668c29cb6c542 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=beca8c14a62748248cfa365d75f1e371]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:56:40.098337 11338 tablet_service.cc:1467] Processing CreateTablet for tablet 6f4f6dfbb2f04447971668c29cb6c542 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=beca8c14a62748248cfa365d75f1e371]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:56:40.098510 10935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f4f6dfbb2f04447971668c29cb6c542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:40.098517 11338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f4f6dfbb2f04447971668c29cb6c542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:40.099879 11465 tablet_bootstrap.cc:492] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be: Bootstrap starting.
I20260808 16:56:40.099987 11133 tablet_service.cc:1467] Processing CreateTablet for tablet 6f4f6dfbb2f04447971668c29cb6c542 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=beca8c14a62748248cfa365d75f1e371]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:56:40.100126 11133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f4f6dfbb2f04447971668c29cb6c542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:40.100368 11465 tablet_bootstrap.cc:654] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:40.101006 11465 tablet_bootstrap.cc:492] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be: No bootstrap required, opened a new log
I20260808 16:56:40.101075 11465 ts_tablet_manager.cc:1397] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:40.101228 11465 raft_consensus.cc:348] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.101306 11465 raft_consensus.cc:374] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:40.101337 11465 raft_consensus.cc:729] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90f814c78a441ca8417c9e052e726be, State: Initialized, Role: FOLLOWER
I20260808 16:56:40.101364 11468 tablet_bootstrap.cc:492] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4: Bootstrap starting.
I20260808 16:56:40.101375 11465 consensus_queue.cc:260] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.101402 11471 tablet_bootstrap.cc:492] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3: Bootstrap starting.
I20260808 16:56:40.101476 11465 ts_tablet_manager.cc:1428] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:40.101521 11473 tablet_replica.cc:439] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } } }
I20260808 16:56:40.101609 11473 tablet_replica.cc:442] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:56:40.101812 11471 tablet_bootstrap.cc:654] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:40.102067 11468 tablet_bootstrap.cc:654] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:40.102568 11471 tablet_bootstrap.cc:492] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3: No bootstrap required, opened a new log
I20260808 16:56:40.102646 11471 ts_tablet_manager.cc:1397] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 16:56:40.102813 11471 raft_consensus.cc:348] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.102887 11471 raft_consensus.cc:374] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:40.102931 11471 raft_consensus.cc:729] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b451aecf307d4e7eb08540ba338eb1d3, State: Initialized, Role: FOLLOWER
I20260808 16:56:40.102962 11468 tablet_bootstrap.cc:492] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4: No bootstrap required, opened a new log
I20260808 16:56:40.102988 11471 consensus_queue.cc:260] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.103011 11468 ts_tablet_manager.cc:1397] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:56:40.103088 11471 ts_tablet_manager.cc:1428] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 16:56:40.103137 11475 tablet_replica.cc:439] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } } }
I20260808 16:56:40.103202 11475 tablet_replica.cc:442] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:56:40.103163 11468 raft_consensus.cc:348] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.103250 11468 raft_consensus.cc:374] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:40.103287 11468 raft_consensus.cc:729] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c498df9fc364f6b8876f6c89e20c3b4, State: Initialized, Role: FOLLOWER
I20260808 16:56:40.103340 11468 consensus_queue.cc:260] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.103452 11468 ts_tablet_manager.cc:1428] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:40.103585 11474 tablet_replica.cc:439] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } } }
I20260808 16:56:40.103665 11474 tablet_replica.cc:442] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:56:40.147567 11475 raft_consensus.cc:482] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:56:40.147663 11475 raft_consensus.cc:504] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.147835 11475 leader_election.cc:288] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c498df9fc364f6b8876f6c89e20c3b4 (127.8.14.66:38605), e90f814c78a441ca8417c9e052e726be (127.8.14.67:46385)
I20260808 16:56:40.148072 11365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f4f6dfbb2f04447971668c29cb6c542" candidate_uuid: "b451aecf307d4e7eb08540ba338eb1d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90f814c78a441ca8417c9e052e726be" is_pre_election: true
I20260808 16:56:40.148073 11164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f4f6dfbb2f04447971668c29cb6c542" candidate_uuid: "b451aecf307d4e7eb08540ba338eb1d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" is_pre_election: true
I20260808 16:56:40.148167 11164 raft_consensus.cc:2454] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b451aecf307d4e7eb08540ba338eb1d3 in term 0.
I20260808 16:56:40.148167 11365 raft_consensus.cc:2454] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b451aecf307d4e7eb08540ba338eb1d3 in term 0.
I20260808 16:56:40.148310 10871 leader_election.cc:302] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c498df9fc364f6b8876f6c89e20c3b4, b451aecf307d4e7eb08540ba338eb1d3; no voters:
I20260808 16:56:40.148442 11475 raft_consensus.cc:2781] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:56:40.148490 11475 raft_consensus.cc:482] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:56:40.148521 11475 raft_consensus.cc:3037] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:40.149185 11475 raft_consensus.cc:504] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.149340 11475 leader_election.cc:288] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [CANDIDATE]: Term 1 election: Requested vote from peers 5c498df9fc364f6b8876f6c89e20c3b4 (127.8.14.66:38605), e90f814c78a441ca8417c9e052e726be (127.8.14.67:46385)
I20260808 16:56:40.149483 11365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f4f6dfbb2f04447971668c29cb6c542" candidate_uuid: "b451aecf307d4e7eb08540ba338eb1d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90f814c78a441ca8417c9e052e726be"
I20260808 16:56:40.149525 11164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f4f6dfbb2f04447971668c29cb6c542" candidate_uuid: "b451aecf307d4e7eb08540ba338eb1d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c498df9fc364f6b8876f6c89e20c3b4"
I20260808 16:56:40.149567 11365 raft_consensus.cc:3037] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:40.149595 11164 raft_consensus.cc:3037] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:40.150321 11365 raft_consensus.cc:2454] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b451aecf307d4e7eb08540ba338eb1d3 in term 1.
I20260808 16:56:40.150315 11164 raft_consensus.cc:2454] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b451aecf307d4e7eb08540ba338eb1d3 in term 1.
I20260808 16:56:40.150492 10871 leader_election.cc:302] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c498df9fc364f6b8876f6c89e20c3b4, b451aecf307d4e7eb08540ba338eb1d3; no voters:
I20260808 16:56:40.150635 11475 raft_consensus.cc:2781] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:40.150687 11475 raft_consensus.cc:686] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [term 1 LEADER]: Becoming Leader. State: Replica: b451aecf307d4e7eb08540ba338eb1d3, State: Running, Role: LEADER
I20260808 16:56:40.150738 11475 consensus_queue.cc:237] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.151032 11477 tablet_replica.cc:439] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3: TxnStatusTablet state changed. Reason: New leader b451aecf307d4e7eb08540ba338eb1d3. Latest consensus state: current_term: 1 leader_uuid: "b451aecf307d4e7eb08540ba338eb1d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } } }
I20260808 16:56:40.151134 11477 tablet_replica.cc:442] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:56:40.151371 11493 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 16:56:40.151337 10732 catalog_manager.cc:5495] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to b451aecf307d4e7eb08540ba338eb1d3 (127.8.14.65). New cstate: current_term: 1 leader_uuid: "b451aecf307d4e7eb08540ba338eb1d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } health_report { overall_health: UNKNOWN } } }
I20260808 16:56:40.151475 11493 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 16:56:40.161820 11462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.159426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44600 (local address 127.8.14.65:38483)
0808 16:56:40.159473 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:40.159475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:40.159484 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:56:40.159572 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:40.159573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:40.159592 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:40.159662 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:40.159793 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.160262 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.160997 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.161209 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.161292 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.161489 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:40.161576 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:40.161681 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:40.161732 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 16:56:40.161752 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:56:40.161847 11486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.159358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:38483 (local address 127.0.0.1:44600)
0808 16:56:40.159434 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:40.159447 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:40.159497 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:40.159670 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:40.159671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:40.159672 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:56:40.159721 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.159724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.160353 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.160354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.160909 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.160911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.161306 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.161307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.161370 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.161389 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:56:40.161659 (+ 270us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:56:40.161670 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 16:56:40.161726 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 16:56:40.161776 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 16:56:40.163641 11164 raft_consensus.cc:1261] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 FOLLOWER]: Refusing update from remote peer b451aecf307d4e7eb08540ba338eb1d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:40.163938 11365 raft_consensus.cc:1261] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be [term 1 FOLLOWER]: Refusing update from remote peer b451aecf307d4e7eb08540ba338eb1d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:40.163970 11475 consensus_queue.cc:1035] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:40.164275 11475 consensus_queue.cc:1035] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:40.165234 11474 tablet_replica.cc:439] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4: TxnStatusTablet state changed. Reason: New leader b451aecf307d4e7eb08540ba338eb1d3. Latest consensus state: current_term: 1 leader_uuid: "b451aecf307d4e7eb08540ba338eb1d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } } }
I20260808 16:56:40.165360 11474 tablet_replica.cc:442] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:56:40.165423 11473 tablet_replica.cc:439] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be: TxnStatusTablet state changed. Reason: New leader b451aecf307d4e7eb08540ba338eb1d3. Latest consensus state: current_term: 1 leader_uuid: "b451aecf307d4e7eb08540ba338eb1d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } } }
I20260808 16:56:40.165517 11473 tablet_replica.cc:442] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:56:40.166285 11477 tablet_replica.cc:439] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b451aecf307d4e7eb08540ba338eb1d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } } }
I20260808 16:56:40.166375 11477 tablet_replica.cc:442] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:56:40.166450 11473 tablet_replica.cc:439] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b451aecf307d4e7eb08540ba338eb1d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } } }
I20260808 16:56:40.166554 11473 tablet_replica.cc:442] T 6f4f6dfbb2f04447971668c29cb6c542 P e90f814c78a441ca8417c9e052e726be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:56:40.167106 11499 mvcc.cc:205] Tried to move back new op lower bound from 7316308787867725824 to 7316308787817750528. Current Snapshot: MvccSnapshot[applied={T|T < 7316308787867725824 or (T in {7316308787867725824})}]
I20260808 16:56:40.167817 11477 tablet_replica.cc:439] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b451aecf307d4e7eb08540ba338eb1d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } } }
I20260808 16:56:40.167905 11477 tablet_replica.cc:442] T 6f4f6dfbb2f04447971668c29cb6c542 P b451aecf307d4e7eb08540ba338eb1d3: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:56:40.168298 11498 mvcc.cc:205] Tried to move back new op lower bound from 7316308787867725824 to 7316308787817750528. Current Snapshot: MvccSnapshot[applied={T|T < 7316308787867725824}]
I20260808 16:56:40.168386 11474 tablet_replica.cc:439] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b451aecf307d4e7eb08540ba338eb1d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } } }
I20260808 16:56:40.168445 11474 tablet_replica.cc:442] T 6f4f6dfbb2f04447971668c29cb6c542 P 5c498df9fc364f6b8876f6c89e20c3b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:56:40.171838 11462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.169191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44614 (local address 127.8.14.65:38483)
0808 16:56:40.169228 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:40.169230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:40.169239 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:56:40.169310 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:40.169311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:40.169318 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:40.169360 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:40.169493 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.170030 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.171148 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.171430 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.171493 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.171646 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:40.171669 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:40.171703 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:40.171776 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 16:56:40.171791 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 16:56:40.176772 11462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.174344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:43707 (local address 127.8.14.65:38483)
0808 16:56:40.174384 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:40.174386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:40.174611 (+ 225us) server_negotiation.cc:407] Connection header received
0808 16:56:40.174670 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:40.174670 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:40.174679 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:40.174731 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:40.174873 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.175319 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.176062 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.176248 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.176338 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.176524 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:40.176552 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:40.176622 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:40.176673 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 16:56:40.176705 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":22}
I20260808 16:56:40.176853 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.174255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:38483 (local address 127.8.14.65:43707)
0808 16:56:40.174535 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:40.174543 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:40.174621 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:40.174733 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:40.174733 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:40.174734 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:56:40.174791 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.174792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.175402 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.175403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.175970 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.175971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.176340 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.176341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.176415 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.176433 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:56:40.176623 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:56:40.176625 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:56:40.176671 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 16:56:40.176728 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":14,"thread_start_us":83,"threads_started":1}
I20260808 16:56:40.179095 11365 raft_consensus.cc:1261] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be [term 1 FOLLOWER]: Refusing update from remote peer b451aecf307d4e7eb08540ba338eb1d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:40.179154 11159 raft_consensus.cc:1261] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 FOLLOWER]: Refusing update from remote peer b451aecf307d4e7eb08540ba338eb1d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:40.179359 11513 consensus_queue.cc:1035] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:40.179492 11475 consensus_queue.cc:1035] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:40.180708 11497 mvcc.cc:205] Tried to move back new op lower bound from 7316308787932086272 to 7316308787335155712. Current Snapshot: MvccSnapshot[applied={T|T < 7316308787932086272}]
I20260808 16:56:40.193557 10734 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:56:40.193661 10734 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:56:40.194362 10734 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3: Sending DeleteTablet for 3 replicas of tablet dae3b496ea2544579fb42dcbc756a772
I20260808 16:56:40.194702 10932 tablet_service.cc:1514] Processing DeleteTablet for tablet dae3b496ea2544579fb42dcbc756a772 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:40 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:44594
I20260808 16:56:40.194756 11338 tablet_service.cc:1514] Processing DeleteTablet for tablet dae3b496ea2544579fb42dcbc756a772 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:40 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:43338
I20260808 16:56:40.194754 11133 tablet_service.cc:1514] Processing DeleteTablet for tablet dae3b496ea2544579fb42dcbc756a772 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:40 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:47560
I20260808 16:56:40.195024 11519 tablet_replica.cc:330] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3: stopping tablet replica
I20260808 16:56:40.195035 11522 tablet_replica.cc:330] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4: stopping tablet replica
I20260808 16:56:40.195129 11522 raft_consensus.cc:2229] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:56:40.195163 11519 raft_consensus.cc:2229] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:56:40.195237 11522 raft_consensus.cc:2258] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:40.195183 10734 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 16:56:40.195300 11519 raft_consensus.cc:2258] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 16:56:40.195324 10734 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:56:40.195890 11519 ts_tablet_manager.cc:1905] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:40.195912 11522 ts_tablet_manager.cc:1905] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:40.196341 10932 tablet_service.cc:1467] Processing CreateTablet for tablet 302c1c79b8844a2b8f5f69fc6b546340 (DEFAULT_TABLE table=test-table [id=a284d74f1cc1494eb44e85b22842907e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:40.196455 10932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302c1c79b8844a2b8f5f69fc6b546340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:40.197695 11522 ts_tablet_manager.cc:1918] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:56:40.197726 11519 ts_tablet_manager.cc:1918] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:56:40.197764 11522 log.cc:1196] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/dae3b496ea2544579fb42dcbc756a772
I20260808 16:56:40.197785 11519 log.cc:1196] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/wal/wals/dae3b496ea2544579fb42dcbc756a772
I20260808 16:56:40.197985 11471 tablet_bootstrap.cc:492] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3: Bootstrap starting.
I20260808 16:56:40.198045 11519 ts_tablet_manager.cc:1939] T dae3b496ea2544579fb42dcbc756a772 P b451aecf307d4e7eb08540ba338eb1d3: Deleting consensus metadata
I20260808 16:56:40.198045 11522 ts_tablet_manager.cc:1939] T dae3b496ea2544579fb42dcbc756a772 P 5c498df9fc364f6b8876f6c89e20c3b4: Deleting consensus metadata
I20260808 16:56:40.198385 11471 tablet_bootstrap.cc:654] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:40.198791 10712 catalog_manager.cc:4841] TS b451aecf307d4e7eb08540ba338eb1d3 (127.8.14.65:38483): tablet dae3b496ea2544579fb42dcbc756a772 (table test-table [id=008ca597472b470195afba2b58e13cec]) successfully deleted
I20260808 16:56:40.199072 10714 catalog_manager.cc:4841] TS 5c498df9fc364f6b8876f6c89e20c3b4 (127.8.14.66:38605): tablet dae3b496ea2544579fb42dcbc756a772 (table test-table [id=008ca597472b470195afba2b58e13cec]) successfully deleted
I20260808 16:56:40.199255 11471 tablet_bootstrap.cc:492] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3: No bootstrap required, opened a new log
I20260808 16:56:40.199285 11133 tablet_service.cc:1467] Processing CreateTablet for tablet 302c1c79b8844a2b8f5f69fc6b546340 (DEFAULT_TABLE table=test-table [id=a284d74f1cc1494eb44e85b22842907e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:40.199311 11471 ts_tablet_manager.cc:1397] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 16:56:40.199352 11338 tablet_service.cc:1467] Processing CreateTablet for tablet 302c1c79b8844a2b8f5f69fc6b546340 (DEFAULT_TABLE table=test-table [id=a284d74f1cc1494eb44e85b22842907e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:40.199394 11133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302c1c79b8844a2b8f5f69fc6b546340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:40.199445 11338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302c1c79b8844a2b8f5f69fc6b546340. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:40.199434 11471 raft_consensus.cc:348] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.199493 11471 raft_consensus.cc:374] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:40.199513 11471 raft_consensus.cc:729] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b451aecf307d4e7eb08540ba338eb1d3, State: Initialized, Role: FOLLOWER
I20260808 16:56:40.199553 11471 consensus_queue.cc:260] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.199630 11471 ts_tablet_manager.cc:1428] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:40.200577 11468 tablet_bootstrap.cc:492] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4: Bootstrap starting.
I20260808 16:56:40.200766 11465 tablet_bootstrap.cc:492] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be: Bootstrap starting.
I20260808 16:56:40.200996 11468 tablet_bootstrap.cc:654] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:40.201144 11465 tablet_bootstrap.cc:654] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:40.201655 11468 tablet_bootstrap.cc:492] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4: No bootstrap required, opened a new log
I20260808 16:56:40.201702 11468 ts_tablet_manager.cc:1397] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:40.201748 11523 tablet_replica.cc:330] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be: stopping tablet replica
I20260808 16:56:40.201835 11465 tablet_bootstrap.cc:492] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be: No bootstrap required, opened a new log
I20260808 16:56:40.201830 11468 raft_consensus.cc:348] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.201841 11523 raft_consensus.cc:2229] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:56:40.201889 11468 raft_consensus.cc:374] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:40.201886 11465 ts_tablet_manager.cc:1397] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:40.201910 11468 raft_consensus.cc:729] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c498df9fc364f6b8876f6c89e20c3b4, State: Initialized, Role: FOLLOWER
I20260808 16:56:40.201922 11523 raft_consensus.cc:2258] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:40.201948 11468 consensus_queue.cc:260] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.202025 11468 ts_tablet_manager.cc:1428] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:40.202018 11465 raft_consensus.cc:348] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.202076 11465 raft_consensus.cc:374] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:40.202098 11465 raft_consensus.cc:729] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90f814c78a441ca8417c9e052e726be, State: Initialized, Role: FOLLOWER
I20260808 16:56:40.202157 11465 consensus_queue.cc:260] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.202250 11465 ts_tablet_manager.cc:1428] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:40.202617 11523 ts_tablet_manager.cc:1905] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:40.204072 11523 ts_tablet_manager.cc:1918] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:56:40.204133 11523 log.cc:1196] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/dae3b496ea2544579fb42dcbc756a772
I20260808 16:56:40.204392 11523 ts_tablet_manager.cc:1939] T dae3b496ea2544579fb42dcbc756a772 P e90f814c78a441ca8417c9e052e726be: Deleting consensus metadata
I20260808 16:56:40.204802 10711 catalog_manager.cc:4841] TS e90f814c78a441ca8417c9e052e726be (127.8.14.67:46385): tablet dae3b496ea2544579fb42dcbc756a772 (table test-table [id=008ca597472b470195afba2b58e13cec]) successfully deleted
I20260808 16:56:40.410808 11474 raft_consensus.cc:482] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:56:40.410944 11474 raft_consensus.cc:504] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.411316 11474 leader_election.cc:288] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b451aecf307d4e7eb08540ba338eb1d3 (127.8.14.65:38483), e90f814c78a441ca8417c9e052e726be (127.8.14.67:46385)
I20260808 16:56:40.415422 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.411597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:44495 (local address 127.8.14.67:46385)
0808 16:56:40.411645 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:40.411647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:40.411751 (+ 104us) server_negotiation.cc:407] Connection header received
0808 16:56:40.411836 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:40.411837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:40.411847 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:40.411928 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:40.412188 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.412932 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.414451 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.415063 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.415124 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.415133 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:40.415311 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 16:56:40.415351 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 16:56:40.415422 11462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.411585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:35169 (local address 127.8.14.65:38483)
0808 16:56:40.411642 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:40.411645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:40.411819 (+ 174us) server_negotiation.cc:407] Connection header received
0808 16:56:40.411868 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:40.411869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:40.411877 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:40.411932 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:40.412182 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.412927 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.414457 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.415019 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.415091 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.415106 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:40.415316 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 16:56:40.415351 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 16:56:40.415491 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.411514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:46385 (local address 127.8.14.66:44495)
0808 16:56:40.411640 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:40.411652 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:40.411751 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:40.411955 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:40.411956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:40.411958 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:40.412092 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.412093 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.413035 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.413035 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.414342 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.414345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.415135 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.415136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.415229 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.415234 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:56:40.415292 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 16:56:40.415341 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":34,"thread_start_us":62,"threads_started":1}
I20260808 16:56:40.415503 11531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.411514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:38483 (local address 127.8.14.66:35169)
0808 16:56:40.411746 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:40.411756 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:40.411818 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:40.411955 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:40.411956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:40.411960 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:40.412082 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.412085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.413034 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.413035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.414337 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.414341 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.415134 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.415135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.415234 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.415237 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:56:40.415292 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 16:56:40.415341 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":90,"thread_start_us":47,"threads_started":1}
I20260808 16:56:40.415807 10963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302c1c79b8844a2b8f5f69fc6b546340" candidate_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b451aecf307d4e7eb08540ba338eb1d3" is_pre_election: true
I20260808 16:56:40.415923 10963 raft_consensus.cc:2454] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c498df9fc364f6b8876f6c89e20c3b4 in term 0.
I20260808 16:56:40.416049 11365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302c1c79b8844a2b8f5f69fc6b546340" candidate_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90f814c78a441ca8417c9e052e726be" is_pre_election: true
I20260808 16:56:40.416152 11365 raft_consensus.cc:2454] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c498df9fc364f6b8876f6c89e20c3b4 in term 0.
I20260808 16:56:40.416163 11062 leader_election.cc:302] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c498df9fc364f6b8876f6c89e20c3b4, b451aecf307d4e7eb08540ba338eb1d3; no voters:
I20260808 16:56:40.416319 11474 raft_consensus.cc:2781] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:56:40.416365 11474 raft_consensus.cc:482] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:56:40.416386 11474 raft_consensus.cc:3037] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:40.417042 11474 raft_consensus.cc:504] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.417182 11474 leader_election.cc:288] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [CANDIDATE]: Term 1 election: Requested vote from peers b451aecf307d4e7eb08540ba338eb1d3 (127.8.14.65:38483), e90f814c78a441ca8417c9e052e726be (127.8.14.67:46385)
I20260808 16:56:40.417323 10963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302c1c79b8844a2b8f5f69fc6b546340" candidate_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b451aecf307d4e7eb08540ba338eb1d3"
I20260808 16:56:40.417403 10963 raft_consensus.cc:3037] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:40.417361 11365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302c1c79b8844a2b8f5f69fc6b546340" candidate_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90f814c78a441ca8417c9e052e726be"
I20260808 16:56:40.417462 11365 raft_consensus.cc:3037] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:40.418036 10963 raft_consensus.cc:2454] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c498df9fc364f6b8876f6c89e20c3b4 in term 1.
I20260808 16:56:40.418040 11365 raft_consensus.cc:2454] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c498df9fc364f6b8876f6c89e20c3b4 in term 1.
I20260808 16:56:40.418222 11062 leader_election.cc:302] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c498df9fc364f6b8876f6c89e20c3b4, b451aecf307d4e7eb08540ba338eb1d3; no voters:
I20260808 16:56:40.418332 11474 raft_consensus.cc:2781] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:40.418467 11474 raft_consensus.cc:686] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 LEADER]: Becoming Leader. State: Replica: 5c498df9fc364f6b8876f6c89e20c3b4, State: Running, Role: LEADER
I20260808 16:56:40.418543 11474 consensus_queue.cc:237] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:40.419288 10729 catalog_manager.cc:5495] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c498df9fc364f6b8876f6c89e20c3b4 (127.8.14.66). New cstate: current_term: 1 leader_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } health_report { overall_health: UNKNOWN } } }
I20260808 16:56:40.433202 11464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.431000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47568 (local address 127.8.14.66:38605)
0808 16:56:40.431039 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:40.431041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:40.431097 (+ 56us) server_negotiation.cc:407] Connection header received
0808 16:56:40.431153 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:40.431154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:40.431170 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:40.431231 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:40.431332 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.431774 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.432525 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.432706 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.432766 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.432902 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:40.433007 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:40.433063 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:40.433132 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 16:56:40.433150 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:56:40.437742 11462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.435602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:55485 (local address 127.8.14.65:38483)
0808 16:56:40.435666 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:40.435667 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:40.435762 (+ 95us) server_negotiation.cc:407] Connection header received
0808 16:56:40.435788 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:40.435789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:40.435799 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:40.435848 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:40.435966 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.436396 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.437173 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.437353 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.437401 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.437557 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:40.437583 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:40.437620 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:40.437678 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 16:56:40.437695 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 16:56:40.437765 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.435517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:38483 (local address 127.8.14.66:55485)
0808 16:56:40.435685 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:40.435695 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:40.435754 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:40.435872 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:40.435873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:40.435874 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:56:40.435926 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.435927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.436457 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.436457 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.437095 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.437098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.437422 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.437423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.437471 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.437484 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:56:40.437640 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:56:40.437641 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 16:56:40.437675 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 16:56:40.437709 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":103,"threads_started":1}
I20260808 16:56:40.439987 11365 raft_consensus.cc:1261] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be [term 1 FOLLOWER]: Refusing update from remote peer 5c498df9fc364f6b8876f6c89e20c3b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:40.440127 10963 raft_consensus.cc:1261] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3 [term 1 FOLLOWER]: Refusing update from remote peer 5c498df9fc364f6b8876f6c89e20c3b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:40.440454 11532 consensus_queue.cc:1035] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:40.440598 11474 consensus_queue.cc:1035] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:40.448526 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.445965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:38605 (local address 127.8.14.65:45961)
0808 16:56:40.446012 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:40.446022 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:40.446059 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:40.446246 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:40.446247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:40.446249 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:56:40.446290 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.446292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.447077 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.447078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.447859 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:40.447862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.448163 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.448163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:40.448194 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.448199 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:56:40.448354 (+ 155us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:56:40.448356 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:56:40.448401 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 16:56:40.448441 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 16:56:40.448553 11464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:40.446023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:45961 (local address 127.8.14.66:38605)
0808 16:56:40.446059 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:40.446061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:40.446070 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:56:40.446114 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:40.446115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:40.446150 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:40.446209 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:40.446366 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.446853 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.447917 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:40.448088 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:40.448163 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:40.448257 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:40.448284 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:40.448333 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:40.448471 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 16:56:40.448491 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 16:56:40.448844 11498 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 16:56:41.969292 10734 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:56:41.969396 10734 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:56:41.970273 10734 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3: Sending DeleteTablet for 3 replicas of tablet 302c1c79b8844a2b8f5f69fc6b546340
I20260808 16:56:41.970575 10932 tablet_service.cc:1514] Processing DeleteTablet for tablet 302c1c79b8844a2b8f5f69fc6b546340 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:41 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:44594
I20260808 16:56:41.970559 11133 tablet_service.cc:1514] Processing DeleteTablet for tablet 302c1c79b8844a2b8f5f69fc6b546340 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:41 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:47560
I20260808 16:56:41.970760 11338 tablet_service.cc:1514] Processing DeleteTablet for tablet 302c1c79b8844a2b8f5f69fc6b546340 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:41 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:43338
I20260808 16:56:41.970970 11573 tablet_replica.cc:330] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be: stopping tablet replica
I20260808 16:56:41.971037 11573 raft_consensus.cc:2229] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:56:41.971079 11573 raft_consensus.cc:2258] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:41.971300 11573 ts_tablet_manager.cc:1905] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:41.972755 11573 ts_tablet_manager.cc:1918] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 16:56:41.972832 11573 log.cc:1196] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/302c1c79b8844a2b8f5f69fc6b546340
I20260808 16:56:41.973064 11573 ts_tablet_manager.cc:1939] T 302c1c79b8844a2b8f5f69fc6b546340 P e90f814c78a441ca8417c9e052e726be: Deleting consensus metadata
I20260808 16:56:41.973381 10711 catalog_manager.cc:4841] TS e90f814c78a441ca8417c9e052e726be (127.8.14.67:46385): tablet 302c1c79b8844a2b8f5f69fc6b546340 (table test-table [id=a284d74f1cc1494eb44e85b22842907e]) successfully deleted
I20260808 16:56:41.973758 11572 tablet_replica.cc:330] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3: stopping tablet replica
I20260808 16:56:41.973851 11572 raft_consensus.cc:2229] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:56:41.973896 11572 raft_consensus.cc:2258] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:41.973996 11571 tablet_replica.cc:330] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4: stopping tablet replica
I20260808 16:56:41.974062 11571 raft_consensus.cc:2229] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:56:41.974110 11572 ts_tablet_manager.cc:1905] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:41.974175 11571 raft_consensus.cc:2258] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:41.974433 11571 ts_tablet_manager.cc:1905] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:41.975777 11571 ts_tablet_manager.cc:1918] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 16:56:41.975845 11571 log.cc:1196] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/302c1c79b8844a2b8f5f69fc6b546340
I20260808 16:56:41.976085 11571 ts_tablet_manager.cc:1939] T 302c1c79b8844a2b8f5f69fc6b546340 P 5c498df9fc364f6b8876f6c89e20c3b4: Deleting consensus metadata
I20260808 16:56:41.976121 11572 ts_tablet_manager.cc:1918] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 16:56:41.976198 11572 log.cc:1196] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/wal/wals/302c1c79b8844a2b8f5f69fc6b546340
I20260808 16:56:41.976388 10714 catalog_manager.cc:4841] TS 5c498df9fc364f6b8876f6c89e20c3b4 (127.8.14.66:38605): tablet 302c1c79b8844a2b8f5f69fc6b546340 (table test-table [id=a284d74f1cc1494eb44e85b22842907e]) successfully deleted
I20260808 16:56:41.976437 11572 ts_tablet_manager.cc:1939] T 302c1c79b8844a2b8f5f69fc6b546340 P b451aecf307d4e7eb08540ba338eb1d3: Deleting consensus metadata
I20260808 16:56:41.976768 10712 catalog_manager.cc:4841] TS b451aecf307d4e7eb08540ba338eb1d3 (127.8.14.65:38483): tablet 302c1c79b8844a2b8f5f69fc6b546340 (table test-table [id=a284d74f1cc1494eb44e85b22842907e]) successfully deleted
Aug 08 16:56:41 dist-test-slave-1nl6 krb5kdc[10670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208201, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 16:56:41 dist-test-slave-1nl6 krb5kdc[10670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208201, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:41.995882 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:41.988815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47634 (local address 127.8.14.126:44293)
0808 16:56:41.988955 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:41.988957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:41.989000 (+ 43us) server_negotiation.cc:407] Connection header received
0808 16:56:41.989045 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:41.989046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:41.989057 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:41.989144 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:41.989670 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:41.990163 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:41.991093 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:41.991279 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:41.991341 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:41.993431 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:41.993432 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:41.993432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:41.993439 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:41.994880 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:41.995226 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:41.995227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:41.995227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:41.995281 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:41.995499 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:41.995499 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:41.995500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:41.995600 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:41.995674 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:41.995755 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 16:56:41.995819 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":90,"threads_started":1}
I20260808 16:56:41.997260 10734 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:56:41.997406 10734 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:56:41.998700 10932 tablet_service.cc:1467] Processing CreateTablet for tablet 899625429b114fbab963f19eafed0c2f (DEFAULT_TABLE table=test-table [id=53a3c1d4e5bb403fa0d5504a9a1dc588]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:41.998714 11133 tablet_service.cc:1467] Processing CreateTablet for tablet 899625429b114fbab963f19eafed0c2f (DEFAULT_TABLE table=test-table [id=53a3c1d4e5bb403fa0d5504a9a1dc588]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:41.998836 11133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 899625429b114fbab963f19eafed0c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:41.998836 10932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 899625429b114fbab963f19eafed0c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:41.998838 11338 tablet_service.cc:1467] Processing CreateTablet for tablet 899625429b114fbab963f19eafed0c2f (DEFAULT_TABLE table=test-table [id=53a3c1d4e5bb403fa0d5504a9a1dc588]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:41.999011 11338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 899625429b114fbab963f19eafed0c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:42.000083 11585 tablet_bootstrap.cc:492] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be: Bootstrap starting.
I20260808 16:56:42.000280 11590 tablet_bootstrap.cc:492] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4: Bootstrap starting.
I20260808 16:56:42.000512 11585 tablet_bootstrap.cc:654] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:42.000612 11590 tablet_bootstrap.cc:654] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:42.000908 11586 tablet_bootstrap.cc:492] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3: Bootstrap starting.
I20260808 16:56:42.001155 11585 tablet_bootstrap.cc:492] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be: No bootstrap required, opened a new log
I20260808 16:56:42.001212 11585 ts_tablet_manager.cc:1397] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:42.001302 11586 tablet_bootstrap.cc:654] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:42.001359 11585 raft_consensus.cc:348] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.001451 11585 raft_consensus.cc:374] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:42.001478 11585 raft_consensus.cc:729] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90f814c78a441ca8417c9e052e726be, State: Initialized, Role: FOLLOWER
I20260808 16:56:42.001538 11585 consensus_queue.cc:260] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.001685 11585 ts_tablet_manager.cc:1428] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:42.001973 11586 tablet_bootstrap.cc:492] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3: No bootstrap required, opened a new log
I20260808 16:56:42.002051 11586 ts_tablet_manager.cc:1397] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:42.002128 11590 tablet_bootstrap.cc:492] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4: No bootstrap required, opened a new log
I20260808 16:56:42.002228 11590 ts_tablet_manager.cc:1397] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:56:42.002192 11586 raft_consensus.cc:348] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.002265 11586 raft_consensus.cc:374] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:42.002286 11586 raft_consensus.cc:729] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b451aecf307d4e7eb08540ba338eb1d3, State: Initialized, Role: FOLLOWER
I20260808 16:56:42.002352 11586 consensus_queue.cc:260] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.002359 11590 raft_consensus.cc:348] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.002458 11586 ts_tablet_manager.cc:1428] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:42.002518 11590 raft_consensus.cc:374] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:42.002578 11590 raft_consensus.cc:729] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c498df9fc364f6b8876f6c89e20c3b4, State: Initialized, Role: FOLLOWER
I20260808 16:56:42.002676 11590 consensus_queue.cc:260] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.002946 11590 ts_tablet_manager.cc:1428] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:42.169174 11543 raft_consensus.cc:482] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:56:42.169307 11543 raft_consensus.cc:504] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.169504 11543 leader_election.cc:288] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b451aecf307d4e7eb08540ba338eb1d3 (127.8.14.65:38483), e90f814c78a441ca8417c9e052e726be (127.8.14.67:46385)
I20260808 16:56:42.169734 11365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "899625429b114fbab963f19eafed0c2f" candidate_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90f814c78a441ca8417c9e052e726be" is_pre_election: true
I20260808 16:56:42.169750 10963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "899625429b114fbab963f19eafed0c2f" candidate_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b451aecf307d4e7eb08540ba338eb1d3" is_pre_election: true
I20260808 16:56:42.169832 11365 raft_consensus.cc:2454] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c498df9fc364f6b8876f6c89e20c3b4 in term 0.
I20260808 16:56:42.169871 10963 raft_consensus.cc:2454] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c498df9fc364f6b8876f6c89e20c3b4 in term 0.
I20260808 16:56:42.170017 11061 leader_election.cc:302] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c498df9fc364f6b8876f6c89e20c3b4, e90f814c78a441ca8417c9e052e726be; no voters:
I20260808 16:56:42.170115 11543 raft_consensus.cc:2781] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:56:42.170181 11543 raft_consensus.cc:482] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:56:42.170219 11543 raft_consensus.cc:3037] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:42.171026 11543 raft_consensus.cc:504] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.171182 11543 leader_election.cc:288] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [CANDIDATE]: Term 1 election: Requested vote from peers b451aecf307d4e7eb08540ba338eb1d3 (127.8.14.65:38483), e90f814c78a441ca8417c9e052e726be (127.8.14.67:46385)
I20260808 16:56:42.171360 11365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "899625429b114fbab963f19eafed0c2f" candidate_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e90f814c78a441ca8417c9e052e726be"
I20260808 16:56:42.171362 10963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "899625429b114fbab963f19eafed0c2f" candidate_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b451aecf307d4e7eb08540ba338eb1d3"
I20260808 16:56:42.171439 10963 raft_consensus.cc:3037] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:42.171439 11365 raft_consensus.cc:3037] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:42.172235 10963 raft_consensus.cc:2454] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c498df9fc364f6b8876f6c89e20c3b4 in term 1.
I20260808 16:56:42.172235 11365 raft_consensus.cc:2454] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c498df9fc364f6b8876f6c89e20c3b4 in term 1.
I20260808 16:56:42.172412 11062 leader_election.cc:302] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c498df9fc364f6b8876f6c89e20c3b4, b451aecf307d4e7eb08540ba338eb1d3; no voters:
I20260808 16:56:42.172542 11543 raft_consensus.cc:2781] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:42.172631 11543 raft_consensus.cc:686] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 LEADER]: Becoming Leader. State: Replica: 5c498df9fc364f6b8876f6c89e20c3b4, State: Running, Role: LEADER
I20260808 16:56:42.172689 11543 consensus_queue.cc:237] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.173331 10729 catalog_manager.cc:5495] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c498df9fc364f6b8876f6c89e20c3b4 (127.8.14.66). New cstate: current_term: 1 leader_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } health_report { overall_health: UNKNOWN } } }
I20260808 16:56:42.183449 11598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:42.181313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47574 (local address 127.8.14.66:38605)
0808 16:56:42.181446 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:42.181448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:42.181456 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:56:42.181486 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:42.181487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:42.181496 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:42.181571 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:42.181673 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.182143 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.182832 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.183092 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.183166 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:42.183277 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:42.183304 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:42.183338 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:42.183377 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 16:56:42.183396 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":91,"threads_started":1}
I20260808 16:56:42.185400 11365 raft_consensus.cc:1261] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be [term 1 FOLLOWER]: Refusing update from remote peer 5c498df9fc364f6b8876f6c89e20c3b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:42.185429 10963 raft_consensus.cc:1261] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3 [term 1 FOLLOWER]: Refusing update from remote peer 5c498df9fc364f6b8876f6c89e20c3b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:42.185590 11543 consensus_queue.cc:1035] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:42.185681 11560 consensus_queue.cc:1035] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:42.194948 10734 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:56:42.195060 10734 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:56:42.195606 10734 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3: Sending DeleteTablet for 3 replicas of tablet 899625429b114fbab963f19eafed0c2f
I20260808 16:56:42.195832 10932 tablet_service.cc:1514] Processing DeleteTablet for tablet 899625429b114fbab963f19eafed0c2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:42 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:44594
I20260808 16:56:42.195871 11133 tablet_service.cc:1514] Processing DeleteTablet for tablet 899625429b114fbab963f19eafed0c2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:42 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:47560
I20260808 16:56:42.195928 11572 tablet_replica.cc:330] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3: stopping tablet replica
I20260808 16:56:42.195957 11571 tablet_replica.cc:330] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4: stopping tablet replica
I20260808 16:56:42.195986 11572 raft_consensus.cc:2229] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:56:42.195997 11571 raft_consensus.cc:2229] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:56:42.196031 11572 raft_consensus.cc:2258] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:42.196043 11571 raft_consensus.cc:2258] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:42.196167 11338 tablet_service.cc:1514] Processing DeleteTablet for tablet 899625429b114fbab963f19eafed0c2f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:42 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:43338
I20260808 16:56:42.196255 11573 tablet_replica.cc:330] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be: stopping tablet replica
I20260808 16:56:42.196302 11572 ts_tablet_manager.cc:1905] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:42.196300 11573 raft_consensus.cc:2229] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:56:42.196357 11573 raft_consensus.cc:2258] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:42.196280 10734 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:56:42.196422 10734 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:56:42.196633 11573 ts_tablet_manager.cc:1905] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:42.196772 11571 ts_tablet_manager.cc:1905] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:42.197933 11572 ts_tablet_manager.cc:1918] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:56:42.197947 11571 ts_tablet_manager.cc:1918] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:56:42.198000 11572 log.cc:1196] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-0/wal/wals/899625429b114fbab963f19eafed0c2f
I20260808 16:56:42.198020 11571 log.cc:1196] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/899625429b114fbab963f19eafed0c2f
I20260808 16:56:42.198215 11573 ts_tablet_manager.cc:1918] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:56:42.198233 11572 ts_tablet_manager.cc:1939] T 899625429b114fbab963f19eafed0c2f P b451aecf307d4e7eb08540ba338eb1d3: Deleting consensus metadata
I20260808 16:56:42.198280 11573 log.cc:1196] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/899625429b114fbab963f19eafed0c2f
I20260808 16:56:42.198288 11571 ts_tablet_manager.cc:1939] T 899625429b114fbab963f19eafed0c2f P 5c498df9fc364f6b8876f6c89e20c3b4: Deleting consensus metadata
I20260808 16:56:42.198522 11573 ts_tablet_manager.cc:1939] T 899625429b114fbab963f19eafed0c2f P e90f814c78a441ca8417c9e052e726be: Deleting consensus metadata
I20260808 16:56:42.198616 10714 catalog_manager.cc:4841] TS 5c498df9fc364f6b8876f6c89e20c3b4 (127.8.14.66:38605): tablet 899625429b114fbab963f19eafed0c2f (table test-table [id=53a3c1d4e5bb403fa0d5504a9a1dc588]) successfully deleted
I20260808 16:56:42.198678 10712 catalog_manager.cc:4841] TS b451aecf307d4e7eb08540ba338eb1d3 (127.8.14.65:38483): tablet 899625429b114fbab963f19eafed0c2f (table test-table [id=53a3c1d4e5bb403fa0d5504a9a1dc588]) successfully deleted
I20260808 16:56:42.198791 10711 catalog_manager.cc:4841] TS e90f814c78a441ca8417c9e052e726be (127.8.14.67:46385): tablet 899625429b114fbab963f19eafed0c2f (table test-table [id=53a3c1d4e5bb403fa0d5504a9a1dc588]) successfully deleted
I20260808 16:56:42.199609 11133 tablet_service.cc:1467] Processing CreateTablet for tablet 248d355d28bc4e7fa4dd75fece5fd87b (DEFAULT_TABLE table=test-table [id=dbabda441f8f4f10924fe7d48d1fb9ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:42.199615 11338 tablet_service.cc:1467] Processing CreateTablet for tablet 248d355d28bc4e7fa4dd75fece5fd87b (DEFAULT_TABLE table=test-table [id=dbabda441f8f4f10924fe7d48d1fb9ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:42.199615 10932 tablet_service.cc:1467] Processing CreateTablet for tablet 248d355d28bc4e7fa4dd75fece5fd87b (DEFAULT_TABLE table=test-table [id=dbabda441f8f4f10924fe7d48d1fb9ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:42.199724 11133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248d355d28bc4e7fa4dd75fece5fd87b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:42.199724 11338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248d355d28bc4e7fa4dd75fece5fd87b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:42.199740 10932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 248d355d28bc4e7fa4dd75fece5fd87b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:42.200577 11586 tablet_bootstrap.cc:492] T 248d355d28bc4e7fa4dd75fece5fd87b P b451aecf307d4e7eb08540ba338eb1d3: Bootstrap starting.
I20260808 16:56:42.200829 11590 tablet_bootstrap.cc:492] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4: Bootstrap starting.
I20260808 16:56:42.200899 11586 tablet_bootstrap.cc:654] T 248d355d28bc4e7fa4dd75fece5fd87b P b451aecf307d4e7eb08540ba338eb1d3: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:42.201166 11585 tablet_bootstrap.cc:492] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be: Bootstrap starting.
I20260808 16:56:42.201222 11590 tablet_bootstrap.cc:654] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:42.201565 11585 tablet_bootstrap.cc:654] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:42.201845 11590 tablet_bootstrap.cc:492] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4: No bootstrap required, opened a new log
I20260808 16:56:42.201891 11590 ts_tablet_manager.cc:1397] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:42.202009 11586 tablet_bootstrap.cc:492] T 248d355d28bc4e7fa4dd75fece5fd87b P b451aecf307d4e7eb08540ba338eb1d3: No bootstrap required, opened a new log
I20260808 16:56:42.202045 11586 ts_tablet_manager.cc:1397] T 248d355d28bc4e7fa4dd75fece5fd87b P b451aecf307d4e7eb08540ba338eb1d3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:42.202011 11590 raft_consensus.cc:348] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.202071 11590 raft_consensus.cc:374] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:42.202105 11590 raft_consensus.cc:729] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c498df9fc364f6b8876f6c89e20c3b4, State: Initialized, Role: FOLLOWER
I20260808 16:56:42.202148 11590 consensus_queue.cc:260] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.202143 11586 raft_consensus.cc:348] T 248d355d28bc4e7fa4dd75fece5fd87b P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.202199 11586 raft_consensus.cc:374] T 248d355d28bc4e7fa4dd75fece5fd87b P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:42.202219 11586 raft_consensus.cc:729] T 248d355d28bc4e7fa4dd75fece5fd87b P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b451aecf307d4e7eb08540ba338eb1d3, State: Initialized, Role: FOLLOWER
I20260808 16:56:42.202234 11590 ts_tablet_manager.cc:1428] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:42.202255 11586 consensus_queue.cc:260] T 248d355d28bc4e7fa4dd75fece5fd87b P b451aecf307d4e7eb08540ba338eb1d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.202358 11586 ts_tablet_manager.cc:1428] T 248d355d28bc4e7fa4dd75fece5fd87b P b451aecf307d4e7eb08540ba338eb1d3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:42.202546 11585 tablet_bootstrap.cc:492] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be: No bootstrap required, opened a new log
I20260808 16:56:42.202602 11585 ts_tablet_manager.cc:1397] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:42.202724 11585 raft_consensus.cc:348] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.202785 11585 raft_consensus.cc:374] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:42.202807 11585 raft_consensus.cc:729] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e90f814c78a441ca8417c9e052e726be, State: Initialized, Role: FOLLOWER
I20260808 16:56:42.202840 11585 consensus_queue.cc:260] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.202965 11585 ts_tablet_manager.cc:1428] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 16:56:42.417862 11594 raft_consensus.cc:482] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:56:42.417992 11594 raft_consensus.cc:504] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.418346 11594 leader_election.cc:288] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b451aecf307d4e7eb08540ba338eb1d3 (127.8.14.65:38483), 5c498df9fc364f6b8876f6c89e20c3b4 (127.8.14.66:38605)
I20260808 16:56:42.421370 11598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:42.418616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:58757 (local address 127.8.14.66:38605)
0808 16:56:42.418673 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:42.418675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:42.418863 (+ 188us) server_negotiation.cc:407] Connection header received
0808 16:56:42.418933 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:42.418934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:42.418945 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:42.419002 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:42.419188 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.419630 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.420632 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.421004 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.421060 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:42.421069 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:42.421231 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 16:56:42.421275 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 16:56:42.421499 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:42.418662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:56143 (local address 127.8.14.65:38483)
0808 16:56:42.418790 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:42.418792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:42.418833 (+ 41us) server_negotiation.cc:407] Connection header received
0808 16:56:42.418863 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:42.418864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:42.418877 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:42.418965 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:42.419136 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.419618 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.420663 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.421111 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.421185 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:42.421199 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:42.421362 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 16:56:42.421422 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":70,"threads_started":1}
I20260808 16:56:42.421403 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:42.418542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:38605 (local address 127.8.14.67:58757)
0808 16:56:42.418748 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:42.418760 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:42.418844 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:42.419019 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:42.419020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:42.419023 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:42.419114 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:42.419117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.419744 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.419745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:42.420547 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:42.420549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.421075 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.421077 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:42.421150 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:42.421155 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:56:42.421227 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 16:56:42.421278 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":21,"thread_start_us":158,"threads_started":1}
I20260808 16:56:42.421551 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:42.418542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:38483 (local address 127.8.14.67:56143)
0808 16:56:42.418708 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:42.418719 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:42.418778 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:42.418969 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:42.418970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:42.418973 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:42.419058 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:42.419061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.419695 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.419696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:42.420551 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:42.420554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.421212 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.421213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:42.421282 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:42.421287 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:56:42.421347 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 16:56:42.421399 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":44,"thread_start_us":88,"threads_started":1}
I20260808 16:56:42.421869 10963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248d355d28bc4e7fa4dd75fece5fd87b" candidate_uuid: "e90f814c78a441ca8417c9e052e726be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b451aecf307d4e7eb08540ba338eb1d3" is_pre_election: true
I20260808 16:56:42.421890 11159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248d355d28bc4e7fa4dd75fece5fd87b" candidate_uuid: "e90f814c78a441ca8417c9e052e726be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" is_pre_election: true
I20260808 16:56:42.421972 10963 raft_consensus.cc:2454] T 248d355d28bc4e7fa4dd75fece5fd87b P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90f814c78a441ca8417c9e052e726be in term 0.
I20260808 16:56:42.421972 11159 raft_consensus.cc:2454] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e90f814c78a441ca8417c9e052e726be in term 0.
I20260808 16:56:42.422143 11267 leader_election.cc:302] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b451aecf307d4e7eb08540ba338eb1d3, e90f814c78a441ca8417c9e052e726be; no voters:
I20260808 16:56:42.422263 11594 raft_consensus.cc:2781] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:56:42.422315 11594 raft_consensus.cc:482] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:56:42.422338 11594 raft_consensus.cc:3037] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:42.423156 11594 raft_consensus.cc:504] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.423293 11594 leader_election.cc:288] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [CANDIDATE]: Term 1 election: Requested vote from peers b451aecf307d4e7eb08540ba338eb1d3 (127.8.14.65:38483), 5c498df9fc364f6b8876f6c89e20c3b4 (127.8.14.66:38605)
I20260808 16:56:42.423427 11159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248d355d28bc4e7fa4dd75fece5fd87b" candidate_uuid: "e90f814c78a441ca8417c9e052e726be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c498df9fc364f6b8876f6c89e20c3b4"
I20260808 16:56:42.423504 11159 raft_consensus.cc:3037] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:42.423467 10963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "248d355d28bc4e7fa4dd75fece5fd87b" candidate_uuid: "e90f814c78a441ca8417c9e052e726be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b451aecf307d4e7eb08540ba338eb1d3"
I20260808 16:56:42.423550 10963 raft_consensus.cc:3037] T 248d355d28bc4e7fa4dd75fece5fd87b P b451aecf307d4e7eb08540ba338eb1d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:42.424194 11159 raft_consensus.cc:2454] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90f814c78a441ca8417c9e052e726be in term 1.
I20260808 16:56:42.424201 10963 raft_consensus.cc:2454] T 248d355d28bc4e7fa4dd75fece5fd87b P b451aecf307d4e7eb08540ba338eb1d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e90f814c78a441ca8417c9e052e726be in term 1.
I20260808 16:56:42.424408 11267 leader_election.cc:302] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b451aecf307d4e7eb08540ba338eb1d3, e90f814c78a441ca8417c9e052e726be; no voters:
I20260808 16:56:42.424525 11594 raft_consensus.cc:2781] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:42.424744 11594 raft_consensus.cc:686] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [term 1 LEADER]: Becoming Leader. State: Replica: e90f814c78a441ca8417c9e052e726be, State: Running, Role: LEADER
I20260808 16:56:42.424836 11594 consensus_queue.cc:237] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } }
I20260808 16:56:42.425498 10734 catalog_manager.cc:5495] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be reported cstate change: term changed from 0 to 1, leader changed from <none> to e90f814c78a441ca8417c9e052e726be (127.8.14.67). New cstate: current_term: 1 leader_uuid: "e90f814c78a441ca8417c9e052e726be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e90f814c78a441ca8417c9e052e726be" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 46385 } health_report { overall_health: HEALTHY } } }
I20260808 16:56:42.486891 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:42.484606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43346 (local address 127.8.14.67:46385)
0808 16:56:42.484805 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:42.484806 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:42.484813 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:56:42.484850 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:42.484850 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:42.484869 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:42.484941 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:42.485048 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.485533 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.486176 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.486375 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.486425 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:42.486544 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:42.486646 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:42.486697 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:42.486730 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 16:56:42.486744 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":133,"threads_started":1}
I20260808 16:56:42.491439 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:42.489533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:59061 (local address 127.8.14.65:38483)
0808 16:56:42.489572 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:42.489574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:42.489580 (+ 6us) server_negotiation.cc:407] Connection header received
0808 16:56:42.489600 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:42.489600 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:42.489611 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:42.489657 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:42.489766 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.490185 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.490858 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.491047 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.491096 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:42.491248 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:42.491271 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:42.491316 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:42.491371 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 16:56:42.491389 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:56:42.491473 11618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:42.489288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:38483 (local address 127.8.14.67:59061)
0808 16:56:42.489451 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:42.489461 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:42.489533 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:42.489672 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:42.489673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:42.489674 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:56:42.489725 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:42.489726 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.490239 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.490239 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:42.490790 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:42.490791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.491129 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.491129 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:42.491172 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:42.491185 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:56:42.491334 (+ 149us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:56:42.491335 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 16:56:42.491370 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 16:56:42.491403 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
I20260808 16:56:42.493551 10963 raft_consensus.cc:1261] T 248d355d28bc4e7fa4dd75fece5fd87b P b451aecf307d4e7eb08540ba338eb1d3 [term 1 FOLLOWER]: Refusing update from remote peer e90f814c78a441ca8417c9e052e726be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:42.493561 11159 raft_consensus.cc:1261] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 FOLLOWER]: Refusing update from remote peer e90f814c78a441ca8417c9e052e726be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:42.493840 11611 consensus_queue.cc:1035] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c498df9fc364f6b8876f6c89e20c3b4" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:42.493964 11594 consensus_queue.cc:1035] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [LEADER]: Connected to new peer: Peer: permanent_uuid: "b451aecf307d4e7eb08540ba338eb1d3" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:42.500909 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:42.498811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:33289 (local address 127.8.14.67:46385)
0808 16:56:42.498854 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:42.498857 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:42.498865 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:56:42.498914 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:42.498915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:42.498926 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:42.498977 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:42.499080 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.499520 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.500218 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.500504 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.500580 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:42.500720 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:42.500752 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:42.500817 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:42.500852 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 16:56:42.500868 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:56:42.500928 11628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:42.498674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:46385 (local address 127.8.14.65:33289)
0808 16:56:42.498798 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:42.498809 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:42.498857 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:42.498984 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:42.498985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:42.498987 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:56:42.499029 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:42.499030 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.499598 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.499599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:42.500138 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:42.500140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:42.500577 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:42.500577 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:42.500643 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:42.500647 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:56:42.500815 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:56:42.500816 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 16:56:42.500843 (+ 27us) client_negotiation.cc:240] Negotiation successful
0808 16:56:42.500875 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":79,"threads_started":1}
W20260808 16:56:42.501199 11551 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 16:56:44.020083 10732 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:56:44.020183 10732 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:56:44.020887 10732 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fed7652b01b34b36a37a97faa7d278e3: Sending DeleteTablet for 3 replicas of tablet 248d355d28bc4e7fa4dd75fece5fd87b
I20260808 16:56:44.021119 10932 tablet_service.cc:1514] Processing DeleteTablet for tablet 248d355d28bc4e7fa4dd75fece5fd87b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:44 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:44594
I20260808 16:56:44.021190 11133 tablet_service.cc:1514] Processing DeleteTablet for tablet 248d355d28bc4e7fa4dd75fece5fd87b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:44 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:47560
I20260808 16:56:44.021190 11338 tablet_service.cc:1514] Processing DeleteTablet for tablet 248d355d28bc4e7fa4dd75fece5fd87b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:44 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:43338
I20260808 16:56:44.021303 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 10846
I20260808 16:56:44.021313 11663 tablet_replica.cc:330] T 248d355d28bc4e7fa4dd75fece5fd87b P b451aecf307d4e7eb08540ba338eb1d3: stopping tablet replica
I20260808 16:56:44.031497 11667 tablet_replica.cc:330] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be: stopping tablet replica
I20260808 16:56:44.031594 11667 raft_consensus.cc:2229] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:56:44.031733 11667 raft_consensus.cc:2258] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:44.032246 11664 tablet_replica.cc:330] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4: stopping tablet replica
I20260808 16:56:44.032320 11664 raft_consensus.cc:2229] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:56:44.032387 11664 raft_consensus.cc:2258] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:44.033128 11664 ts_tablet_manager.cc:1905] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:44.035315 11667 ts_tablet_manager.cc:1905] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:44.035627 11664 ts_tablet_manager.cc:1918] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 16:56:44.035702 11664 log.cc:1196] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/248d355d28bc4e7fa4dd75fece5fd87b
I20260808 16:56:44.035977 11664 ts_tablet_manager.cc:1939] T 248d355d28bc4e7fa4dd75fece5fd87b P 5c498df9fc364f6b8876f6c89e20c3b4: Deleting consensus metadata
I20260808 16:56:44.036908 10714 catalog_manager.cc:4841] TS 5c498df9fc364f6b8876f6c89e20c3b4 (127.8.14.66:38605): tablet 248d355d28bc4e7fa4dd75fece5fd87b (table test-table [id=dbabda441f8f4f10924fe7d48d1fb9ee]) successfully deleted
I20260808 16:56:44.037117 11667 ts_tablet_manager.cc:1918] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 16:56:44.037186 11667 log.cc:1196] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/248d355d28bc4e7fa4dd75fece5fd87b
I20260808 16:56:44.037487 11667 ts_tablet_manager.cc:1939] T 248d355d28bc4e7fa4dd75fece5fd87b P e90f814c78a441ca8417c9e052e726be: Deleting consensus metadata
I20260808 16:56:44.037766 10711 catalog_manager.cc:4841] TS e90f814c78a441ca8417c9e052e726be (127.8.14.67:46385): tablet 248d355d28bc4e7fa4dd75fece5fd87b (table test-table [id=dbabda441f8f4f10924fe7d48d1fb9ee]) successfully deleted
W20260808 16:56:44.046427 10712 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.14.65:38483 (error 108)
I20260808 16:56:44.046717 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 11039
I20260808 16:56:44.051354 11671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:44.046848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:38483 (local address 127.0.0.1:44626)
0808 16:56:44.051240 (+ 4392us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:44.051294 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.65:38483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4364,"thread_start_us":86,"threads_started":1}
W20260808 16:56:44.051612 10712 catalog_manager.cc:4584] TS b451aecf307d4e7eb08540ba338eb1d3 (127.8.14.65:38483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 248d355d28bc4e7fa4dd75fece5fd87b: Network error: Client connection negotiation failed: client connection to 127.8.14.65:38483: connect: Connection refused (error 111)
I20260808 16:56:44.057542 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 11239
I20260808 16:56:44.070729 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 10686
2026-08-08T16:56:44Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6215 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:44 dist-test-slave-1nl6 krb5kdc[11675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:44 dist-test-slave-1nl6 krb5kdc[11675](info): set up 2 sockets
Aug 08 16:56:44 dist-test-slave-1nl6 krb5kdc[11675](info): commencing operation
krb5kdc: starting...
W20260808 16:56:45.312084 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:56:45 dist-test-slave-1nl6 krb5kdc[11675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:56:45.427173 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:41843
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:43719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:41843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:45.508244 11694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:45.508484 11694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:45.508524 11694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:45.509641 11694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:45.509711 11694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:45.509735 11694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:45.509758 11694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:45.509784 11694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:45.511267 11694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:43719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:41843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:41843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:45.511540 11694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:45.511761 11694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:45.516822 11707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:45.516845 11694 server_base.cc:1034] running on GCE node
W20260808 16:56:45.516803 11705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:45.516765 11703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:45.517315 11694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:45.517598 11694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:45.518756 11694 hybrid_clock.cc:648] HybridClock initialized: now 1786208205518750 us; error 35 us; skew 500 ppm
Aug 08 16:56:45 dist-test-slave-1nl6 krb5kdc[11675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208205, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:45.521139 11694 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:56:45.521564 11694 webserver.cc:460] Webserver started at http://127.8.14.126:39031/ using document root <none> and password file <none>
I20260808 16:56:45.521759 11694 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:45.521818 11694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:45.521905 11694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:45.522825 11694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "6c1a238351264b81a0eea576080638b9"
format_stamp: "Formatted at 2026-08-08 16:56:45 on dist-test-slave-1nl6"
server_key: "1cd18ac6537a9741d6cc69c0c57f8f27"
server_key_iv: "3e1f892746f6fb32cbfb796b2df5e6c9"
server_key_version: "encryptionkey@0"
I20260808 16:56:45.523205 11694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "6c1a238351264b81a0eea576080638b9"
format_stamp: "Formatted at 2026-08-08 16:56:45 on dist-test-slave-1nl6"
server_key: "1cd18ac6537a9741d6cc69c0c57f8f27"
server_key_iv: "3e1f892746f6fb32cbfb796b2df5e6c9"
server_key_version: "encryptionkey@0"
I20260808 16:56:45.524452 11694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:45.525111 11714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:45.525267 11694 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:45.525357 11694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "6c1a238351264b81a0eea576080638b9"
format_stamp: "Formatted at 2026-08-08 16:56:45 on dist-test-slave-1nl6"
server_key: "1cd18ac6537a9741d6cc69c0c57f8f27"
server_key_iv: "3e1f892746f6fb32cbfb796b2df5e6c9"
server_key_version: "encryptionkey@0"
I20260808 16:56:45.525411 11694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:45.541522 11694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:45.543769 11694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:45.543888 11694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:45.547662 11694 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:41843
I20260808 16:56:45.547708 11780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:41843 every 8 connection(s)
I20260808 16:56:45.548055 11694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:45.548923 11784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:45.551126 11784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9: Bootstrap starting.
I20260808 16:56:45.551692 11784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:45.551998 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 11694
I20260808 16:56:45.552002 11784 log.cc:824] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:45.552074 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:56:45.552256 8249 external_mini_cluster.cc:1428] Setting key 36fba0ec7950bd6bfce643eaef55a50d
I20260808 16:56:45.552749 11784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9: No bootstrap required, opened a new log
I20260808 16:56:45.554772 11784 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c1a238351264b81a0eea576080638b9" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41843 } }
I20260808 16:56:45.554966 11784 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:45.555011 11784 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1a238351264b81a0eea576080638b9, State: Initialized, Role: FOLLOWER
I20260808 16:56:45.555158 11784 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c1a238351264b81a0eea576080638b9" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41843 } }
I20260808 16:56:45.555250 11784 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:45.555290 11784 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:45.555337 11784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:45.555974 11784 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c1a238351264b81a0eea576080638b9" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41843 } }
I20260808 16:56:45.556079 11784 leader_election.cc:302] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c1a238351264b81a0eea576080638b9; no voters:
I20260808 16:56:45.556226 11784 leader_election.cc:288] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:45.556291 11789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:45.556452 11784 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:45.556486 11789 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [term 1 LEADER]: Becoming Leader. State: Replica: 6c1a238351264b81a0eea576080638b9, State: Running, Role: LEADER
I20260808 16:56:45.556576 11789 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c1a238351264b81a0eea576080638b9" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41843 } }
I20260808 16:56:45.556938 11789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c1a238351264b81a0eea576080638b9. Latest consensus state: current_term: 1 leader_uuid: "6c1a238351264b81a0eea576080638b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c1a238351264b81a0eea576080638b9" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41843 } } }
I20260808 16:56:45.557188 11789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:45.557116 11790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c1a238351264b81a0eea576080638b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c1a238351264b81a0eea576080638b9" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41843 } } }
I20260808 16:56:45.557600 11790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:45.557709 11802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:45.558214 11802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:45.559856 11802 catalog_manager.cc:1347] Generated new cluster ID: 2e7ccf4e25824a3aa6230e0041d317d4
I20260808 16:56:45.559909 11802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:56:45 dist-test-slave-1nl6 krb5kdc[11675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:45.565387 11788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:45.552730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58990 (local address 127.8.14.126:41843)
0808 16:56:45.552927 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:45.552937 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:45.552969 (+ 32us) server_negotiation.cc:407] Connection header received
0808 16:56:45.553985 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:45.553994 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:45.554054 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:45.554211 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:45.554773 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:45.556505 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:45.557732 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:45.558431 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:45.558656 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:45.561315 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:45.561320 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:45.561320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:45.561332 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:45.563363 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:45.563747 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:45.563750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:45.563750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:45.563861 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:45.564067 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:45.564068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:45.564068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:45.564389 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:45.564956 (+ 567us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:45.565041 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 16:56:45.565179 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260808 16:56:45.573031 11802 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:45.573491 11802 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:45.575845 11802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c1a238351264b81a0eea576080638b9: Generated new TSK 0
I20260808 16:56:45.576005 11802 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 16:56:45.603322 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:41843
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:43719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:45.684448 11825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:45.684644 11825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:45.684666 11825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:45.685745 11825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:45.685775 11825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:45.685801 11825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:56:45.687176 11825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:43719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:41843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:45.687369 11825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:45.687585 11825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:45.692183 11837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:45.692238 11832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:45.692243 11834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:45.692277 11825 server_base.cc:1034] running on GCE node
I20260808 16:56:45.692543 11825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:45.692788 11825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:45.693948 11825 hybrid_clock.cc:648] HybridClock initialized: now 1786208205693935 us; error 25 us; skew 500 ppm
Aug 08 16:56:45 dist-test-slave-1nl6 krb5kdc[11675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208205, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:45.696376 11825 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:56:45.696792 11825 webserver.cc:460] Webserver started at http://127.8.14.65:33431/ using document root <none> and password file <none>
I20260808 16:56:45.697044 11825 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:45.697106 11825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:45.697248 11825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:45.698246 11825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "17ce0bcb5eb04257a498031f8f78e534"
format_stamp: "Formatted at 2026-08-08 16:56:45 on dist-test-slave-1nl6"
server_key: "09544c124b622f64286fce6783d0feee"
server_key_iv: "763af3381fd38b110231dbfad3ca490a"
server_key_version: "encryptionkey@0"
I20260808 16:56:45.698589 11825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "17ce0bcb5eb04257a498031f8f78e534"
format_stamp: "Formatted at 2026-08-08 16:56:45 on dist-test-slave-1nl6"
server_key: "09544c124b622f64286fce6783d0feee"
server_key_iv: "763af3381fd38b110231dbfad3ca490a"
server_key_version: "encryptionkey@0"
I20260808 16:56:45.699962 11825 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:45.700692 11848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:45.700853 11825 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:45.700953 11825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "17ce0bcb5eb04257a498031f8f78e534"
format_stamp: "Formatted at 2026-08-08 16:56:45 on dist-test-slave-1nl6"
server_key: "09544c124b622f64286fce6783d0feee"
server_key_iv: "763af3381fd38b110231dbfad3ca490a"
server_key_version: "encryptionkey@0"
I20260808 16:56:45.701023 11825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:45.729900 11825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:45.731712 11825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:45.731879 11825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:45.732120 11825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:45.732429 11825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:45.732481 11825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:45.732522 11825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:45.732550 11825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:45.738245 11825 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:46299
I20260808 16:56:45.738315 12027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:46299 every 8 connection(s)
I20260808 16:56:45.738626 11825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
Aug 08 16:56:45 dist-test-slave-1nl6 krb5kdc[11675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208205, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:45.747531 11788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:45.739355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:52343 (local address 127.8.14.126:41843)
0808 16:56:45.739408 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:45.739410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:45.739611 (+ 201us) server_negotiation.cc:407] Connection header received
0808 16:56:45.740110 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:45.740111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:45.740122 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:45.740177 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:45.741093 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:45.741611 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:45.742576 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:45.742787 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:45.742871 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:45.744950 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:45.744952 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:45.744952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:45.744960 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:45.746140 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:45.746544 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:45.746546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:45.746546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:45.746601 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:45.746873 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:45.746875 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:45.746875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:45.747016 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:45.747087 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:45.747351 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 16:56:45.747441 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 16:56:45.747731 12030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:45.739339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:41843 (local address 127.8.14.65:52343)
0808 16:56:45.739504 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:45.739538 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:45.740012 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:45.740245 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:45.740249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:45.740803 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:45.740990 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:45.740995 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:45.741711 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:45.741712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:45.742486 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:45.742488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:45.742883 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:45.742884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:45.743032 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:45.743513 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:45.743521 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:45.744859 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:45.746239 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:45.746247 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:45.746249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:45.746461 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:45.746674 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:45.746675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:45.746675 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:45.746804 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:45.747089 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:45.747097 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:45.747227 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 16:56:45.747329 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 16:56:45.747479 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260808 16:56:45.748193 12028 heartbeater.cc:347] Connected to a master server at 127.8.14.126:41843
I20260808 16:56:45.748288 12028 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:45.748396 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 11825
I20260808 16:56:45.748466 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:56:45.748477 12028 heartbeater.cc:511] Master 127.8.14.126:41843 requested a full tablet report, sending...
I20260808 16:56:45.748647 8249 external_mini_cluster.cc:1428] Setting key 237e66386148054e0245e44da9fad4c4
I20260808 16:56:45.748972 11737 ts_manager.cc:194] Registered new tserver with Master: 17ce0bcb5eb04257a498031f8f78e534 (127.8.14.65:46299)
I20260808 16:56:45.749585 11737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:52343
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 16:56:45.771538 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:41843
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:43719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:45.852705 12034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:45.852892 12034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:45.852916 12034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:45.854061 12034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:45.854105 12034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:45.854163 12034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:56:45.855649 12034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:43719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:41843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:45.855876 12034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:45.856087 12034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:45.861251 12044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:45.861274 12043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:45.861320 12034 server_base.cc:1034] running on GCE node
W20260808 16:56:45.861286 12048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:45.861613 12034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:45.861853 12034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:45.862993 12034 hybrid_clock.cc:648] HybridClock initialized: now 1786208205862965 us; error 36 us; skew 500 ppm
Aug 08 16:56:45 dist-test-slave-1nl6 krb5kdc[11675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208205, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:45.865309 12034 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:56:45.865761 12034 webserver.cc:460] Webserver started at http://127.8.14.66:33857/ using document root <none> and password file <none>
I20260808 16:56:45.866009 12034 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:45.866075 12034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:45.866205 12034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:45.867280 12034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "b2f4ed1eea284f6b9241d844bc48fb2c"
format_stamp: "Formatted at 2026-08-08 16:56:45 on dist-test-slave-1nl6"
server_key: "64b745dfddfc209fe4c02d26db5ee90f"
server_key_iv: "92f42e83c5966ebd130fe218a7cceca4"
server_key_version: "encryptionkey@0"
I20260808 16:56:45.867635 12034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "b2f4ed1eea284f6b9241d844bc48fb2c"
format_stamp: "Formatted at 2026-08-08 16:56:45 on dist-test-slave-1nl6"
server_key: "64b745dfddfc209fe4c02d26db5ee90f"
server_key_iv: "92f42e83c5966ebd130fe218a7cceca4"
server_key_version: "encryptionkey@0"
I20260808 16:56:45.868927 12034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:56:45.869616 12062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:45.869786 12034 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:45.869872 12034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "b2f4ed1eea284f6b9241d844bc48fb2c"
format_stamp: "Formatted at 2026-08-08 16:56:45 on dist-test-slave-1nl6"
server_key: "64b745dfddfc209fe4c02d26db5ee90f"
server_key_iv: "92f42e83c5966ebd130fe218a7cceca4"
server_key_version: "encryptionkey@0"
I20260808 16:56:45.869947 12034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:45.910079 12034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:45.912042 12034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:45.912245 12034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:45.912484 12034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:45.912813 12034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:45.912865 12034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:45.912909 12034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:45.912938 12034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:45.918632 12034 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:41621
I20260808 16:56:45.918715 12234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:41621 every 8 connection(s)
I20260808 16:56:45.919065 12034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
Aug 08 16:56:45 dist-test-slave-1nl6 krb5kdc[11675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208205, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:45.926517 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 12034
I20260808 16:56:45.926625 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:56:45.926854 8249 external_mini_cluster.cc:1428] Setting key 4e9d6ff5f7d60ab5ceea070cf174c325
I20260808 16:56:45.928036 11788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:45.919776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:41027 (local address 127.8.14.126:41843)
0808 16:56:45.919826 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:45.919828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:45.920043 (+ 215us) server_negotiation.cc:407] Connection header received
0808 16:56:45.920512 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:45.920513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:45.920523 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:45.920573 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:45.921473 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:45.921958 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:45.922997 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:45.923213 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:45.923283 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:45.925336 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:45.925337 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:45.925338 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:45.925345 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:45.926531 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:45.927061 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:45.927063 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:45.927063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:45.927131 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:45.927395 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:45.927396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:45.927397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:45.927531 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:45.927616 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:45.927866 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 16:56:45.927949 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:56:45.928210 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:45.919764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:41843 (local address 127.8.14.66:41027)
0808 16:56:45.919927 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:45.919963 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:45.920434 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:45.920624 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:45.920628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:45.921199 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:45.921397 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:45.921402 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:45.922076 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:45.922077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:45.922877 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:45.922879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:45.923293 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:45.923294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:45.923403 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:45.923882 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:45.923890 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:45.925243 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:45.926623 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:45.926630 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:45.926633 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:45.926976 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:45.927206 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:45.927207 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:45.927208 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:45.927341 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:45.927615 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:45.927625 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:45.927734 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 16:56:45.927838 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 16:56:45.927988 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260808 16:56:45.928711 12235 heartbeater.cc:347] Connected to a master server at 127.8.14.126:41843
I20260808 16:56:45.928799 12235 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:45.928985 12235 heartbeater.cc:511] Master 127.8.14.126:41843 requested a full tablet report, sending...
I20260808 16:56:45.929405 11737 ts_manager.cc:194] Registered new tserver with Master: b2f4ed1eea284f6b9241d844bc48fb2c (127.8.14.66:41621)
I20260808 16:56:45.929839 11737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:41027
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 16:56:45.951536 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:41843
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:43719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:46.032374 12245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:46.032563 12245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:46.032586 12245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:46.033660 12245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:46.033694 12245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:46.033720 12245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:56:46.035053 12245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:43719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:41843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:46.035220 12245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:46.035386 12245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:46.040266 12255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:46.040287 12258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:46.040266 12254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:46.040483 12245 server_base.cc:1034] running on GCE node
I20260808 16:56:46.040681 12245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:46.040932 12245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:46.042083 12245 hybrid_clock.cc:648] HybridClock initialized: now 1786208206042072 us; error 33 us; skew 500 ppm
Aug 08 16:56:46 dist-test-slave-1nl6 krb5kdc[11675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208206, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:46.044530 12245 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:56:46.045190 12245 webserver.cc:460] Webserver started at http://127.8.14.67:37515/ using document root <none> and password file <none>
I20260808 16:56:46.045452 12245 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:46.045521 12245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:46.045678 12245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:46.046727 12245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "c9097a9b949544b6b9c8751936d470f7"
format_stamp: "Formatted at 2026-08-08 16:56:46 on dist-test-slave-1nl6"
server_key: "343b7420a975da656a89d38a1a5a001f"
server_key_iv: "6580c3c25f1b71ef28e949f7e17de317"
server_key_version: "encryptionkey@0"
I20260808 16:56:46.047112 12245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "c9097a9b949544b6b9c8751936d470f7"
format_stamp: "Formatted at 2026-08-08 16:56:46 on dist-test-slave-1nl6"
server_key: "343b7420a975da656a89d38a1a5a001f"
server_key_iv: "6580c3c25f1b71ef28e949f7e17de317"
server_key_version: "encryptionkey@0"
I20260808 16:56:46.048516 12245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:56:46.049269 12268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:46.049448 12245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:56:46.049542 12245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "c9097a9b949544b6b9c8751936d470f7"
format_stamp: "Formatted at 2026-08-08 16:56:46 on dist-test-slave-1nl6"
server_key: "343b7420a975da656a89d38a1a5a001f"
server_key_iv: "6580c3c25f1b71ef28e949f7e17de317"
server_key_version: "encryptionkey@0"
I20260808 16:56:46.049615 12245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:46.068953 12245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:46.070683 12245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:46.070837 12245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:46.071069 12245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:46.071370 12245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:46.071422 12245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:46.071462 12245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:46.071491 12245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:46.077567 12245 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:40293
I20260808 16:56:46.077611 12430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:40293 every 8 connection(s)
I20260808 16:56:46.077913 12245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
Aug 08 16:56:46 dist-test-slave-1nl6 krb5kdc[11675](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208206, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:46.086205 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 12245
I20260808 16:56:46.086306 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:56:46.086526 8249 external_mini_cluster.cc:1428] Setting key 1e115e0a835ff04f40a3f9a030702a35
I20260808 16:56:46.086663 11788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:46.078548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:50281 (local address 127.8.14.126:41843)
0808 16:56:46.078599 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:46.078602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:46.078773 (+ 171us) server_negotiation.cc:407] Connection header received
0808 16:56:46.079372 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:46.079374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:46.079387 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:46.079437 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:46.080336 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:46.080818 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:46.081742 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:46.081937 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:46.082020 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:46.084056 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:46.084058 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:46.084058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:46.084066 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:46.085262 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:46.085653 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:46.085654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:46.085654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:46.085714 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:46.085957 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:46.085958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:46.085958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:46.086105 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:46.086184 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:46.086482 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 16:56:46.086572 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 16:56:46.086795 12433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:46.078526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:41843 (local address 127.8.14.67:50281)
0808 16:56:46.078666 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:46.078700 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:46.079261 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:46.079501 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:46.079505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:46.080057 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:46.080232 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:46.080236 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:46.080909 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:46.080910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:46.081652 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:46.081654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:46.082030 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:46.082032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:46.082181 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:46.082611 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:46.082619 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:46.083961 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:46.085363 (+ 1402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:46.085369 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:46.085371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:46.085579 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:46.085777 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:46.085778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:46.085778 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:46.085885 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:46.086184 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:46.086190 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:46.086309 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 16:56:46.086438 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 16:56:46.086557 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
I20260808 16:56:46.087275 12431 heartbeater.cc:347] Connected to a master server at 127.8.14.126:41843
I20260808 16:56:46.087390 12431 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:46.087587 12431 heartbeater.cc:511] Master 127.8.14.126:41843 requested a full tablet report, sending...
I20260808 16:56:46.087936 11737 ts_manager.cc:194] Registered new tserver with Master: c9097a9b949544b6b9c8751936d470f7 (127.8.14.67:40293)
I20260808 16:56:46.088250 11737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:50281
I20260808 16:56:46.098527 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 16:56:46.111986 11788 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:46.111095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59000 (local address 127.8.14.126:41843)
0808 16:56:46.111171 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:46.111174 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:46.111272 (+ 98us) server_negotiation.cc:407] Connection header received
0808 16:56:46.111321 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:46.111322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:46.111332 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:46.111380 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:46.111901 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59000: BlockingRecv error: recv got EOF from 127.0.0.1:59000 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 16:56:46.112879 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 11825
I20260808 16:56:46.129251 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 12034
I20260808 16:56:46.136744 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 12245
I20260808 16:56:46.144080 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 11694
2026-08-08T16:56:46Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2072 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:46 dist-test-slave-1nl6 krb5kdc[12446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:46 dist-test-slave-1nl6 krb5kdc[12446](info): set up 2 sockets
Aug 08 16:56:46 dist-test-slave-1nl6 krb5kdc[12446](info): commencing operation
krb5kdc: starting...
W20260808 16:56:47.364373 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:56:47 dist-test-slave-1nl6 krb5kdc[12446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:56:47.479277 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:33949
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:45633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:33949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:47.560822 12466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:47.561010 12466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:47.561031 12466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:47.562107 12466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:47.562143 12466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:47.562157 12466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:47.562170 12466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:47.562181 12466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:47.563741 12466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:45633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:33949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:33949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:47.563946 12466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:47.564112 12466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:47.568826 12478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:47.568822 12473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:47.568851 12474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:47.569044 12466 server_base.cc:1034] running on GCE node
I20260808 16:56:47.569347 12466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:47.569635 12466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:47.570778 12466 hybrid_clock.cc:648] HybridClock initialized: now 1786208207570759 us; error 37 us; skew 500 ppm
Aug 08 16:56:47 dist-test-slave-1nl6 krb5kdc[12446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208207, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:47.573221 12466 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:56:47.573645 12466 webserver.cc:460] Webserver started at http://127.8.14.126:41935/ using document root <none> and password file <none>
I20260808 16:56:47.573890 12466 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:47.573958 12466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:47.574107 12466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:47.575214 12466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "e32c9aaa7c024874ab5c4b9b507f65b4"
format_stamp: "Formatted at 2026-08-08 16:56:47 on dist-test-slave-1nl6"
server_key: "6a0a977b7c97ed89e6d3a75c223721e4"
server_key_iv: "7abc2f140fab5487123083a3a14a40db"
server_key_version: "encryptionkey@0"
I20260808 16:56:47.575580 12466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "e32c9aaa7c024874ab5c4b9b507f65b4"
format_stamp: "Formatted at 2026-08-08 16:56:47 on dist-test-slave-1nl6"
server_key: "6a0a977b7c97ed89e6d3a75c223721e4"
server_key_iv: "7abc2f140fab5487123083a3a14a40db"
server_key_version: "encryptionkey@0"
I20260808 16:56:47.577000 12466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:47.577772 12488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:47.577936 12466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:47.578047 12466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "e32c9aaa7c024874ab5c4b9b507f65b4"
format_stamp: "Formatted at 2026-08-08 16:56:47 on dist-test-slave-1nl6"
server_key: "6a0a977b7c97ed89e6d3a75c223721e4"
server_key_iv: "7abc2f140fab5487123083a3a14a40db"
server_key_version: "encryptionkey@0"
I20260808 16:56:47.578119 12466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:47.594731 12466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:47.596525 12466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:47.596683 12466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:47.600665 12466 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:33949
I20260808 16:56:47.600701 12556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:33949 every 8 connection(s)
I20260808 16:56:47.601042 12466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:47.601928 12557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:47.604040 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 12466
I20260808 16:56:47.604121 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:56:47.604252 12557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4: Bootstrap starting.
I20260808 16:56:47.604318 8249 external_mini_cluster.cc:1428] Setting key 4020bd5156bdc7a3ccf98d76081d0bce
I20260808 16:56:47.604974 12557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:47.605273 12557 log.cc:824] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:47.606014 12557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4: No bootstrap required, opened a new log
I20260808 16:56:47.608417 12557 raft_consensus.cc:348] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32c9aaa7c024874ab5c4b9b507f65b4" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 33949 } }
I20260808 16:56:47.608596 12557 raft_consensus.cc:374] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:47.608634 12557 raft_consensus.cc:729] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e32c9aaa7c024874ab5c4b9b507f65b4, State: Initialized, Role: FOLLOWER
I20260808 16:56:47.608752 12557 consensus_queue.cc:260] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32c9aaa7c024874ab5c4b9b507f65b4" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 33949 } }
I20260808 16:56:47.608825 12557 raft_consensus.cc:388] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:47.608855 12557 raft_consensus.cc:482] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:47.608901 12557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:47.609637 12557 raft_consensus.cc:504] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32c9aaa7c024874ab5c4b9b507f65b4" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 33949 } }
I20260808 16:56:47.609754 12557 leader_election.cc:302] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e32c9aaa7c024874ab5c4b9b507f65b4; no voters:
I20260808 16:56:47.609905 12557 leader_election.cc:288] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:47.609965 12568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:47.610155 12568 raft_consensus.cc:686] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [term 1 LEADER]: Becoming Leader. State: Replica: e32c9aaa7c024874ab5c4b9b507f65b4, State: Running, Role: LEADER
I20260808 16:56:47.610200 12557 sys_catalog.cc:564] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:47.610358 12568 consensus_queue.cc:237] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32c9aaa7c024874ab5c4b9b507f65b4" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 33949 } }
I20260808 16:56:47.610718 12571 sys_catalog.cc:455] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e32c9aaa7c024874ab5c4b9b507f65b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32c9aaa7c024874ab5c4b9b507f65b4" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 33949 } } }
I20260808 16:56:47.610929 12571 sys_catalog.cc:458] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [sys.catalog]: This master's current role is: LEADER
Aug 08 16:56:47 dist-test-slave-1nl6 krb5kdc[12446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:47.611874 12569 sys_catalog.cc:455] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e32c9aaa7c024874ab5c4b9b507f65b4. Latest consensus state: current_term: 1 leader_uuid: "e32c9aaa7c024874ab5c4b9b507f65b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32c9aaa7c024874ab5c4b9b507f65b4" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 33949 } } }
I20260808 16:56:47.611967 12569 sys_catalog.cc:458] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4 [sys.catalog]: This master's current role is: LEADER
W20260808 16:56:47.612258 12593 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:56:47.612316 12593 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:56:47.612383 12580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:47.612536 12580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:47.614154 12580 catalog_manager.cc:1347] Generated new cluster ID: 394705602b1c442cbdb44e3ea7e67da3
I20260808 16:56:47.614207 12580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:47.616029 12565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:47.604818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54258 (local address 127.8.14.126:33949)
0808 16:56:47.604988 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:47.604996 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:47.605020 (+ 24us) server_negotiation.cc:407] Connection header received
0808 16:56:47.605491 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:47.605498 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:47.605544 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:47.605658 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:47.606353 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:47.607731 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:47.608511 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:47.609153 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:47.609587 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:47.612270 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:47.612274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:47.612275 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:47.612285 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:47.614444 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:47.614810 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:47.614813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:47.614813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:47.614956 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:47.615167 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:47.615168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:47.615168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:47.615402 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:47.615571 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:47.615676 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 16:56:47.615807 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260808 16:56:47.616850 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:47.629956 12580 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:47.630604 12580 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:47.631691 12565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:47.628119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54272 (local address 127.8.14.126:33949)
0808 16:56:47.628156 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:47.628158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:47.628293 (+ 135us) server_negotiation.cc:407] Connection header received
0808 16:56:47.628375 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:47.628376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:47.628385 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:47.628430 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:47.628785 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:47.629267 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:47.630236 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:47.630445 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:47.630521 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:47.631232 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:47.631234 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:47.631234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:47.631406 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:47.631420 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:47.631426 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:47.631489 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:47.631561 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 16:56:47.631594 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:56:47.638033 12580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e32c9aaa7c024874ab5c4b9b507f65b4: Generated new TSK 0
I20260808 16:56:47.638190 12580 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:47.643224 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 12466
2026-08-08T16:56:47Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1505 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:47 dist-test-slave-1nl6 krb5kdc[12612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:47 dist-test-slave-1nl6 krb5kdc[12612](info): set up 2 sockets
Aug 08 16:56:47 dist-test-slave-1nl6 krb5kdc[12612](info): commencing operation
krb5kdc: starting...
W20260808 16:56:48.881935 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:56:48 dist-test-slave-1nl6 krb5kdc[12612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:56:48.999401 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:35163
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:34235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:35163
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:49.081436 12630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:49.081664 12630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:49.081701 12630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:49.082839 12630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:49.082902 12630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:49.082947 12630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:49.082978 12630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:49.083004 12630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:49.084483 12630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:34235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:35163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:35163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:49.084753 12630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:49.084975 12630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:49.090363 12639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:49.090375 12635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:49.090364 12637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:49.090544 12630 server_base.cc:1034] running on GCE node
I20260808 16:56:49.090777 12630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:49.091090 12630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:49.092271 12630 hybrid_clock.cc:648] HybridClock initialized: now 1786208209092242 us; error 41 us; skew 500 ppm
Aug 08 16:56:49 dist-test-slave-1nl6 krb5kdc[12612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:49.094638 12630 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:56:49.095129 12630 webserver.cc:460] Webserver started at http://127.8.14.126:34825/ using document root <none> and password file <none>
I20260808 16:56:49.095350 12630 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:49.095417 12630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:49.095575 12630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:49.096567 12630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "7e1abf7cb35b4aa2a7b707a8c95d4ecb"
format_stamp: "Formatted at 2026-08-08 16:56:49 on dist-test-slave-1nl6"
server_key: "0f590e115988e2452e5b3094b3ae883e"
server_key_iv: "ef49ae61f8162b36c2483c6842f37b15"
server_key_version: "encryptionkey@0"
I20260808 16:56:49.096931 12630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "7e1abf7cb35b4aa2a7b707a8c95d4ecb"
format_stamp: "Formatted at 2026-08-08 16:56:49 on dist-test-slave-1nl6"
server_key: "0f590e115988e2452e5b3094b3ae883e"
server_key_iv: "ef49ae61f8162b36c2483c6842f37b15"
server_key_version: "encryptionkey@0"
I20260808 16:56:49.098253 12630 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:49.099002 12648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:49.099184 12630 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:49.099287 12630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "7e1abf7cb35b4aa2a7b707a8c95d4ecb"
format_stamp: "Formatted at 2026-08-08 16:56:49 on dist-test-slave-1nl6"
server_key: "0f590e115988e2452e5b3094b3ae883e"
server_key_iv: "ef49ae61f8162b36c2483c6842f37b15"
server_key_version: "encryptionkey@0"
I20260808 16:56:49.099376 12630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:49.117409 12630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:49.119207 12630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:49.119366 12630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:49.123251 12630 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:35163
I20260808 16:56:49.123301 12723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:35163 every 8 connection(s)
I20260808 16:56:49.123660 12630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:49.124104 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 12630
I20260808 16:56:49.124220 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:56:49.124511 8249 external_mini_cluster.cc:1428] Setting key 2573243b73a2c86f04711abe9984a214
I20260808 16:56:49.125090 12724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:49.128273 12724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb: Bootstrap starting.
I20260808 16:56:49.128938 12724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:49.129252 12724 log.cc:824] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:49.129992 12724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb: No bootstrap required, opened a new log
I20260808 16:56:49.132359 12724 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e1abf7cb35b4aa2a7b707a8c95d4ecb" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35163 } }
I20260808 16:56:49.132513 12724 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:49.132563 12724 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e1abf7cb35b4aa2a7b707a8c95d4ecb, State: Initialized, Role: FOLLOWER
I20260808 16:56:49.132704 12724 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e1abf7cb35b4aa2a7b707a8c95d4ecb" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35163 } }
I20260808 16:56:49.132818 12724 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:49.132866 12724 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:49.132916 12724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:49.133430 12724 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e1abf7cb35b4aa2a7b707a8c95d4ecb" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35163 } }
I20260808 16:56:49.133550 12724 leader_election.cc:302] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e1abf7cb35b4aa2a7b707a8c95d4ecb; no voters:
I20260808 16:56:49.133730 12724 leader_election.cc:288] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:49.133769 12729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:49.133942 12729 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [term 1 LEADER]: Becoming Leader. State: Replica: 7e1abf7cb35b4aa2a7b707a8c95d4ecb, State: Running, Role: LEADER
I20260808 16:56:49.134029 12724 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:49.134033 12729 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e1abf7cb35b4aa2a7b707a8c95d4ecb" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35163 } }
I20260808 16:56:49.134440 12731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e1abf7cb35b4aa2a7b707a8c95d4ecb. Latest consensus state: current_term: 1 leader_uuid: "7e1abf7cb35b4aa2a7b707a8c95d4ecb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e1abf7cb35b4aa2a7b707a8c95d4ecb" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35163 } } }
I20260808 16:56:49.134585 12731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:49.134476 12730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e1abf7cb35b4aa2a7b707a8c95d4ecb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e1abf7cb35b4aa2a7b707a8c95d4ecb" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35163 } } }
I20260808 16:56:49.135685 12730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb [sys.catalog]: This master's current role is: LEADER
W20260808 16:56:49.136016 12749 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:56:49.136092 12749 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:56:49.136155 12754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:49.136332 12754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 16:56:49 dist-test-slave-1nl6 krb5kdc[12612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:49.137777 12754 catalog_manager.cc:1347] Generated new cluster ID: 55e4a4157310462cb1625afb19d36d08
I20260808 16:56:49.137836 12754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:49.141376 12727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:49.125081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59752 (local address 127.8.14.126:35163)
0808 16:56:49.125259 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:49.125268 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:49.125310 (+ 42us) server_negotiation.cc:407] Connection header received
0808 16:56:49.125842 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:49.125851 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:49.125912 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:49.126060 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:49.126654 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:49.128031 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:49.128845 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:49.129489 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:49.129572 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:49.136962 (+ 7390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:49.136966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:49.136967 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:49.136979 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:49.139219 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:49.139672 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:49.139675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:49.139675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:49.139780 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:49.140011 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:49.140011 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:49.140012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:49.140315 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:49.140470 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:49.141021 (+ 551us) server_negotiation.cc:299] Negotiation successful
0808 16:56:49.141161 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":80,"threads_started":1}
I20260808 16:56:49.142546 12754 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:49.143285 12754 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:49.147590 12754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e1abf7cb35b4aa2a7b707a8c95d4ecb: Generated new TSK 0
I20260808 16:56:49.147761 12754 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 16:56:49.179467 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35163
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:34235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:49.277374 12763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:49.277593 12763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:49.277629 12763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:49.278751 12763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:49.278815 12763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:49.278860 12763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:56:49.280300 12763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:34235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:49.280547 12763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:49.280810 12763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:49.285655 12771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:49.285883 12772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:49.285712 12779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:49.285892 12763 server_base.cc:1034] running on GCE node
I20260808 16:56:49.286252 12763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:49.286561 12763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:49.287750 12763 hybrid_clock.cc:648] HybridClock initialized: now 1786208209287730 us; error 35 us; skew 500 ppm
Aug 08 16:56:49 dist-test-slave-1nl6 krb5kdc[12612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:49.290526 12763 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:56:49.291051 12763 webserver.cc:460] Webserver started at http://127.8.14.65:33485/ using document root <none> and password file <none>
I20260808 16:56:49.291302 12763 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:49.291373 12763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:49.291507 12763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:49.292534 12763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "0d87052d09bc44dab9c9c39afd7559e7"
format_stamp: "Formatted at 2026-08-08 16:56:49 on dist-test-slave-1nl6"
server_key: "abfaac939cd066a9b4fcc6b330a79b14"
server_key_iv: "2b60572e1f3f07f055bf2f480b2e6644"
server_key_version: "encryptionkey@0"
I20260808 16:56:49.292909 12763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "0d87052d09bc44dab9c9c39afd7559e7"
format_stamp: "Formatted at 2026-08-08 16:56:49 on dist-test-slave-1nl6"
server_key: "abfaac939cd066a9b4fcc6b330a79b14"
server_key_iv: "2b60572e1f3f07f055bf2f480b2e6644"
server_key_version: "encryptionkey@0"
I20260808 16:56:49.294279 12763 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:49.295109 12787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:49.295282 12763 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:49.295439 12763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "0d87052d09bc44dab9c9c39afd7559e7"
format_stamp: "Formatted at 2026-08-08 16:56:49 on dist-test-slave-1nl6"
server_key: "abfaac939cd066a9b4fcc6b330a79b14"
server_key_iv: "2b60572e1f3f07f055bf2f480b2e6644"
server_key_version: "encryptionkey@0"
I20260808 16:56:49.295517 12763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:49.315996 12763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:49.318333 12763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:49.318495 12763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:49.318733 12763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:49.319072 12763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:49.319142 12763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:49.319182 12763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:49.319209 12763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:49.324908 12763 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:33375
I20260808 16:56:49.325004 12946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:33375 every 8 connection(s)
I20260808 16:56:49.325309 12763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
Aug 08 16:56:49 dist-test-slave-1nl6 krb5kdc[12612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:49.333666 12727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:49.326026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:46711 (local address 127.8.14.126:35163)
0808 16:56:49.326093 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:49.326095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:49.326313 (+ 218us) server_negotiation.cc:407] Connection header received
0808 16:56:49.326757 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:49.326758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:49.326768 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:49.326829 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:49.327697 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:49.328189 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:49.329112 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:49.329299 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:49.329351 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:49.331386 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:49.331388 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:49.331388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:49.331394 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:49.332566 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:49.332925 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:49.332926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:49.332926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:49.332975 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:49.333180 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:49.333180 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:49.333181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:49.333275 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:49.333327 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:49.333539 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 16:56:49.333602 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 16:56:49.333905 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:49.326002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:35163 (local address 127.8.14.65:46711)
0808 16:56:49.326194 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:49.326228 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:49.326677 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:49.326878 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:49.326881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:49.327435 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:49.327615 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:49.327620 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:49.328267 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:49.328268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:49.329038 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:49.329040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:49.329371 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:49.329372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:49.329465 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:49.329962 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:49.329970 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:49.331307 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:49.332638 (+ 1331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:49.332644 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:49.332645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:49.332860 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:49.333034 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:49.333035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:49.333035 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:49.333126 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:49.333329 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:49.333334 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:49.333434 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 16:56:49.333511 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 16:56:49.333621 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":111,"threads_started":1}
I20260808 16:56:49.334530 12947 heartbeater.cc:347] Connected to a master server at 127.8.14.126:35163
I20260808 16:56:49.334635 12947 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:49.334870 12947 heartbeater.cc:511] Master 127.8.14.126:35163 requested a full tablet report, sending...
I20260808 16:56:49.334980 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 12763
I20260808 16:56:49.335057 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:56:49.335285 8249 external_mini_cluster.cc:1428] Setting key 81d086b9b6fa4c839ed6ec991a8db13e
I20260808 16:56:49.335505 12672 ts_manager.cc:194] Registered new tserver with Master: 0d87052d09bc44dab9c9c39afd7559e7 (127.8.14.65:33375)
I20260808 16:56:49.336251 12672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:46711
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 16:56:49.359022 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35163
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:34235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:49.440660 12958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:49.440881 12958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:49.440919 12958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:49.442067 12958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:49.442138 12958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:49.442185 12958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:56:49.443641 12958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:34235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:49.443900 12958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:49.444136 12958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:49.449123 12973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:49.449106 12971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:49.449260 12958 server_base.cc:1034] running on GCE node
W20260808 16:56:49.449103 12969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:49.449581 12958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:49.449841 12958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:49.450982 12958 hybrid_clock.cc:648] HybridClock initialized: now 1786208209450968 us; error 44 us; skew 500 ppm
Aug 08 16:56:49 dist-test-slave-1nl6 krb5kdc[12612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:49.453512 12958 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:56:49.453953 12958 webserver.cc:460] Webserver started at http://127.8.14.66:36467/ using document root <none> and password file <none>
I20260808 16:56:49.454126 12958 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:49.454165 12958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:49.454252 12958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:49.455264 12958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "217d0ff9fb9543c1a3c30e8719729790"
format_stamp: "Formatted at 2026-08-08 16:56:49 on dist-test-slave-1nl6"
server_key: "4e28a2ee445af22f1079b8a377343a24"
server_key_iv: "ff09121fbd1f72124586fcd9f435842f"
server_key_version: "encryptionkey@0"
I20260808 16:56:49.455569 12958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "217d0ff9fb9543c1a3c30e8719729790"
format_stamp: "Formatted at 2026-08-08 16:56:49 on dist-test-slave-1nl6"
server_key: "4e28a2ee445af22f1079b8a377343a24"
server_key_iv: "ff09121fbd1f72124586fcd9f435842f"
server_key_version: "encryptionkey@0"
I20260808 16:56:49.456789 12958 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:56:49.457450 12982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:49.457626 12958 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:49.457712 12958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "217d0ff9fb9543c1a3c30e8719729790"
format_stamp: "Formatted at 2026-08-08 16:56:49 on dist-test-slave-1nl6"
server_key: "4e28a2ee445af22f1079b8a377343a24"
server_key_iv: "ff09121fbd1f72124586fcd9f435842f"
server_key_version: "encryptionkey@0"
I20260808 16:56:49.457767 12958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:49.464977 12958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:49.467200 12958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:49.467308 12958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:49.467465 12958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:49.467702 12958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:49.467732 12958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:49.467751 12958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:49.467763 12958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:49.473393 12958 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:44347
I20260808 16:56:49.473505 13143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:44347 every 8 connection(s)
I20260808 16:56:49.473788 12958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
Aug 08 16:56:49 dist-test-slave-1nl6 krb5kdc[12612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:49.482792 12727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:49.474513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:59433 (local address 127.8.14.126:35163)
0808 16:56:49.474593 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:49.474596 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:49.474765 (+ 169us) server_negotiation.cc:407] Connection header received
0808 16:56:49.475378 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:49.475380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:49.475392 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:49.475446 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:49.476374 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:49.476870 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:49.477862 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:49.478068 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:49.478148 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:49.480161 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:49.480162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:49.480162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:49.480170 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:49.481578 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:49.481931 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:49.481932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:49.481932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:49.481983 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:49.482219 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:49.482220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:49.482220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:49.482369 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:49.482425 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:49.482656 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 16:56:49.482722 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:56:49.482981 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:49.474473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:35163 (local address 127.8.14.66:59433)
0808 16:56:49.474641 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:49.474685 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:49.475261 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:49.475513 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:49.475517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:49.476087 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:49.476268 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:49.476272 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:49.476976 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:49.476977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:49.477762 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:49.477764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:49.478155 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:49.478156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:49.478290 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:49.478717 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:49.478725 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:49.480070 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:49.481649 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:49.481655 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:49.481657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:49.481865 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:49.482040 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:49.482041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:49.482041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:49.482153 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:49.482430 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:49.482435 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:49.482543 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 16:56:49.482624 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 16:56:49.482741 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260808 16:56:49.483501 13144 heartbeater.cc:347] Connected to a master server at 127.8.14.126:35163
I20260808 16:56:49.483618 13144 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:49.483743 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 12958
I20260808 16:56:49.483836 13144 heartbeater.cc:511] Master 127.8.14.126:35163 requested a full tablet report, sending...
I20260808 16:56:49.483852 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:56:49.484089 8249 external_mini_cluster.cc:1428] Setting key 640288c46e70d8053a5392895d1e100e
I20260808 16:56:49.484251 12672 ts_manager.cc:194] Registered new tserver with Master: 217d0ff9fb9543c1a3c30e8719729790 (127.8.14.66:44347)
I20260808 16:56:49.484824 12672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:59433
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 16:56:49.507894 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35163
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:34235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:49.588438 13150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:49.588665 13150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:49.588701 13150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:49.589825 13150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:49.589890 13150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:49.589932 13150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:56:49.591382 13150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:34235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:49.591640 13150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:49.591861 13150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:56:49.596765 13150 server_base.cc:1034] running on GCE node
W20260808 16:56:49.596745 13155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:49.596732 13156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:49.596737 13158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:49.597195 13150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:49.597455 13150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:49.598594 13150 hybrid_clock.cc:648] HybridClock initialized: now 1786208209598582 us; error 29 us; skew 500 ppm
Aug 08 16:56:49 dist-test-slave-1nl6 krb5kdc[12612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:49.601128 13150 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:56:49.601600 13150 webserver.cc:460] Webserver started at http://127.8.14.67:39155/ using document root <none> and password file <none>
I20260808 16:56:49.601788 13150 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:49.601827 13150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:49.601940 13150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:49.603066 13150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "a3ebff021b7b4013b810b6fa041d345c"
format_stamp: "Formatted at 2026-08-08 16:56:49 on dist-test-slave-1nl6"
server_key: "06d12987434a19d07d4d5a6b725e83bb"
server_key_iv: "3638feec64ca557b92d824eec21fff7a"
server_key_version: "encryptionkey@0"
I20260808 16:56:49.603370 13150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "a3ebff021b7b4013b810b6fa041d345c"
format_stamp: "Formatted at 2026-08-08 16:56:49 on dist-test-slave-1nl6"
server_key: "06d12987434a19d07d4d5a6b725e83bb"
server_key_iv: "3638feec64ca557b92d824eec21fff7a"
server_key_version: "encryptionkey@0"
I20260808 16:56:49.604591 13150 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:56:49.605255 13169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:49.605428 13150 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:56:49.605535 13150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "a3ebff021b7b4013b810b6fa041d345c"
format_stamp: "Formatted at 2026-08-08 16:56:49 on dist-test-slave-1nl6"
server_key: "06d12987434a19d07d4d5a6b725e83bb"
server_key_iv: "3638feec64ca557b92d824eec21fff7a"
server_key_version: "encryptionkey@0"
I20260808 16:56:49.605607 13150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:49.642642 13150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:49.644968 13150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:49.645133 13150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:49.645370 13150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:49.645704 13150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:49.645759 13150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:49.645809 13150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:49.645838 13150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:49.651856 13150 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:46477
I20260808 16:56:49.651902 13329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:46477 every 8 connection(s)
I20260808 16:56:49.652228 13150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 16:56:49.652684 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 13150
I20260808 16:56:49.652796 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:56:49.653028 8249 external_mini_cluster.cc:1428] Setting key 2cfb03ad696033fa576770415874a991
Aug 08 16:56:49 dist-test-slave-1nl6 krb5kdc[12612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:49.661515 12727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:49.652974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:47263 (local address 127.8.14.126:35163)
0808 16:56:49.653046 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:49.653048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:49.653238 (+ 190us) server_negotiation.cc:407] Connection header received
0808 16:56:49.653925 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:49.653926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:49.653935 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:49.653974 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:49.655038 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:49.655875 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:49.656863 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:49.657040 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:49.657091 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:49.659093 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:49.659094 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:49.659095 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:49.659101 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:49.660274 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:49.660679 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:49.660680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:49.660680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:49.660731 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:49.660945 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:49.660945 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:49.660946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:49.661043 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:49.661094 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:49.661329 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 16:56:49.661393 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 16:56:49.661659 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:49.652891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:35163 (local address 127.8.14.67:47263)
0808 16:56:49.653111 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:49.653154 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:49.653830 (+ 676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:49.654035 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:49.654038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:49.654640 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:49.654834 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:49.654839 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:49.655991 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:49.655992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:49.656783 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:49.656785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:49.657111 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:49.657112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:49.657214 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:49.657606 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:49.657613 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:49.659014 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:49.660393 (+ 1379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:49.660399 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:49.660401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:49.660613 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:49.660790 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:49.660791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:49.660791 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:49.660889 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:49.661098 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:49.661102 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:49.661206 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 16:56:49.661283 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 16:56:49.661463 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":106,"threads_started":1}
I20260808 16:56:49.662179 13330 heartbeater.cc:347] Connected to a master server at 127.8.14.126:35163
I20260808 16:56:49.662261 13330 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:49.662395 13330 heartbeater.cc:511] Master 127.8.14.126:35163 requested a full tablet report, sending...
I20260808 16:56:49.662727 12672 ts_manager.cc:194] Registered new tserver with Master: a3ebff021b7b4013b810b6fa041d345c (127.8.14.67:46477)
I20260808 16:56:49.663138 12672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:47263
I20260808 16:56:49.664858 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 16:56:49 dist-test-slave-1nl6 krb5kdc[12612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208209, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 16:56:49 dist-test-slave-1nl6 krb5kdc[12612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208209, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:49.688199 12727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:49.681537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59768 (local address 127.8.14.126:35163)
0808 16:56:49.681607 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:49.681609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:49.681689 (+ 80us) server_negotiation.cc:407] Connection header received
0808 16:56:49.681731 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:49.681731 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:49.681740 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:49.681795 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:49.682312 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:49.682847 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:49.683591 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:49.683765 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:49.683840 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:49.685953 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:49.685954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:49.685955 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:49.685961 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:49.687258 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:49.687558 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:49.687558 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:49.687559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:49.687605 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:49.687762 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:49.687763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:49.687763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:49.687893 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:49.687966 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:49.688049 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 16:56:49.688104 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 16:56:49.688444 12672 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59768
I20260808 16:56:49.688838 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 12763
I20260808 16:56:49.697113 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 12958
I20260808 16:56:49.705446 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 13150
I20260808 16:56:49.717985 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 12630
2026-08-08T16:56:49Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2071 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:49 dist-test-slave-1nl6 krb5kdc[13347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:49 dist-test-slave-1nl6 krb5kdc[13347](info): set up 2 sockets
Aug 08 16:56:49 dist-test-slave-1nl6 krb5kdc[13347](info): commencing operation
krb5kdc: starting...
W20260808 16:56:50.922315 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:56:51 dist-test-slave-1nl6 krb5kdc[13347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:56:51.039050 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:34711
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:34711
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:51.120165 13371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:51.120369 13371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:51.120391 13371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:51.121461 13371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:51.121492 13371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:51.121506 13371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:51.121519 13371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:51.121531 13371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:51.122972 13371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:34711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:34711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:51.123157 13371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:51.123318 13371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:51.128007 13381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:51.127987 13380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:51.127985 13384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:51.128180 13371 server_base.cc:1034] running on GCE node
I20260808 16:56:51.128430 13371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:51.128721 13371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:51.129884 13371 hybrid_clock.cc:648] HybridClock initialized: now 1786208211129876 us; error 37 us; skew 500 ppm
Aug 08 16:56:51 dist-test-slave-1nl6 krb5kdc[13347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208211, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:51.132844 13371 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:56:51.133279 13371 webserver.cc:460] Webserver started at http://127.8.14.126:36115/ using document root <none> and password file <none>
I20260808 16:56:51.133474 13371 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:51.133515 13371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:51.133603 13371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:51.134555 13371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "67d09039dfe4440ab21a1de18a2fe18e"
format_stamp: "Formatted at 2026-08-08 16:56:51 on dist-test-slave-1nl6"
server_key: "93f95e036d272ba4ca6945e78157bed9"
server_key_iv: "0e0a02a3eaa2ebcac7e1c475bb4631e4"
server_key_version: "encryptionkey@0"
I20260808 16:56:51.134862 13371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "67d09039dfe4440ab21a1de18a2fe18e"
format_stamp: "Formatted at 2026-08-08 16:56:51 on dist-test-slave-1nl6"
server_key: "93f95e036d272ba4ca6945e78157bed9"
server_key_iv: "0e0a02a3eaa2ebcac7e1c475bb4631e4"
server_key_version: "encryptionkey@0"
I20260808 16:56:51.136188 13371 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:56:51.136881 13395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:51.137089 13371 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:56:51.137239 13371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "67d09039dfe4440ab21a1de18a2fe18e"
format_stamp: "Formatted at 2026-08-08 16:56:51 on dist-test-slave-1nl6"
server_key: "93f95e036d272ba4ca6945e78157bed9"
server_key_iv: "0e0a02a3eaa2ebcac7e1c475bb4631e4"
server_key_version: "encryptionkey@0"
I20260808 16:56:51.137352 13371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:51.148592 13371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:51.151172 13371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:51.151332 13371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:51.155426 13371 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:34711
I20260808 16:56:51.155457 13464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:34711 every 8 connection(s)
I20260808 16:56:51.155994 13371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:51.156690 13466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:51.159009 13466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e: Bootstrap starting.
I20260808 16:56:51.159606 13466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:51.159916 13466 log.cc:824] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:51.160619 13466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e: No bootstrap required, opened a new log
I20260808 16:56:51.162185 13466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d09039dfe4440ab21a1de18a2fe18e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34711 } }
I20260808 16:56:51.162324 13466 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:51.162374 13466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67d09039dfe4440ab21a1de18a2fe18e, State: Initialized, Role: FOLLOWER
I20260808 16:56:51.162525 13466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d09039dfe4440ab21a1de18a2fe18e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34711 } }
I20260808 16:56:51.162591 13466 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:51.162613 13466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:51.162662 13466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:51.163600 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 13371
I20260808 16:56:51.163625 13466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d09039dfe4440ab21a1de18a2fe18e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34711 } }
I20260808 16:56:51.163691 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:56:51.163745 13466 leader_election.cc:302] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67d09039dfe4440ab21a1de18a2fe18e; no voters:
I20260808 16:56:51.163901 8249 external_mini_cluster.cc:1428] Setting key b9d37429470d018ee0436fcdab7d94f3
I20260808 16:56:51.163959 13466 leader_election.cc:288] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:51.163995 13469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:51.164130 13469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [term 1 LEADER]: Becoming Leader. State: Replica: 67d09039dfe4440ab21a1de18a2fe18e, State: Running, Role: LEADER
I20260808 16:56:51.164224 13466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:51.164212 13469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d09039dfe4440ab21a1de18a2fe18e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34711 } }
I20260808 16:56:51.164813 13472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67d09039dfe4440ab21a1de18a2fe18e. Latest consensus state: current_term: 1 leader_uuid: "67d09039dfe4440ab21a1de18a2fe18e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d09039dfe4440ab21a1de18a2fe18e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34711 } } }
I20260808 16:56:51.164912 13472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:51.165084 13471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67d09039dfe4440ab21a1de18a2fe18e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67d09039dfe4440ab21a1de18a2fe18e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34711 } } }
I20260808 16:56:51.165155 13471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:51.165488 13490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:51.166139 13490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:51.167776 13490 catalog_manager.cc:1347] Generated new cluster ID: 89525c99a4154872ab339496455aca1b
I20260808 16:56:51.167838 13490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:51.171622 13490 catalog_manager.cc:1370] Generated new certificate authority record
Aug 08 16:56:51 dist-test-slave-1nl6 krb5kdc[13347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:51.172343 13490 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:51.175787 13490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67d09039dfe4440ab21a1de18a2fe18e: Generated new TSK 0
I20260808 16:56:51.175953 13490 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:51.176517 13481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:51.164518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51286 (local address 127.8.14.126:34711)
0808 16:56:51.164735 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:51.165344 (+ 609us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:51.165384 (+ 40us) server_negotiation.cc:407] Connection header received
0808 16:56:51.165517 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:51.165526 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:51.165600 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:51.165840 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:51.166642 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.168040 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.169116 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.169778 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.169914 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:51.172681 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:51.172686 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:51.172687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:51.172699 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:51.174734 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:51.175150 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:51.175154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:51.175154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:51.175270 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:51.175502 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:51.175503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:51.175504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:51.175797 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:51.176067 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:51.176141 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 16:56:51.176298 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 16:56:51.201344 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:34711
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:51.282418 13505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:51.282612 13505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:51.282634 13505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:51.283828 13505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:51.283875 13505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:51.283901 13505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:56:51.285446 13505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:34711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:51.285688 13505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:51.285925 13505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:51.291390 13515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:51.291392 13519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:51.291410 13514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:51.291633 13505 server_base.cc:1034] running on GCE node
I20260808 16:56:51.291865 13505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:51.292156 13505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:51.293349 13505 hybrid_clock.cc:648] HybridClock initialized: now 1786208211293318 us; error 39 us; skew 500 ppm
Aug 08 16:56:51 dist-test-slave-1nl6 krb5kdc[13347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208211, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:51.295881 13505 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:56:51.296303 13505 webserver.cc:460] Webserver started at http://127.8.14.65:37069/ using document root <none> and password file <none>
I20260808 16:56:51.296522 13505 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:51.296589 13505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:51.296741 13505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:51.297745 13505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "749fdd71128f4a3c929449596aef607c"
format_stamp: "Formatted at 2026-08-08 16:56:51 on dist-test-slave-1nl6"
server_key: "380648af5308fd88c8fbd49d0f9a0dbb"
server_key_iv: "2d11b3e22172766f190c99a8c7cac71e"
server_key_version: "encryptionkey@0"
I20260808 16:56:51.298111 13505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "749fdd71128f4a3c929449596aef607c"
format_stamp: "Formatted at 2026-08-08 16:56:51 on dist-test-slave-1nl6"
server_key: "380648af5308fd88c8fbd49d0f9a0dbb"
server_key_iv: "2d11b3e22172766f190c99a8c7cac71e"
server_key_version: "encryptionkey@0"
I20260808 16:56:51.299510 13505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:51.300203 13529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:51.300374 13505 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:51.300475 13505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "749fdd71128f4a3c929449596aef607c"
format_stamp: "Formatted at 2026-08-08 16:56:51 on dist-test-slave-1nl6"
server_key: "380648af5308fd88c8fbd49d0f9a0dbb"
server_key_iv: "2d11b3e22172766f190c99a8c7cac71e"
server_key_version: "encryptionkey@0"
I20260808 16:56:51.300546 13505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:51.321969 13505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:51.323860 13505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:51.324024 13505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:51.324258 13505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:51.324584 13505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:51.324640 13505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:51.324679 13505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:51.324710 13505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:51.330376 13505 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:37947
I20260808 16:56:51.330447 13684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:37947 every 8 connection(s)
I20260808 16:56:51.330762 13505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:56:51.336220 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 13505
I20260808 16:56:51.336304 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:56:51.336488 8249 external_mini_cluster.cc:1428] Setting key 122c62857922d7a2e2d1feb725b02791
Aug 08 16:56:51 dist-test-slave-1nl6 krb5kdc[13347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208211, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:51.340000 13481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:51.331490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:39183 (local address 127.8.14.126:34711)
0808 16:56:51.331574 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:51.331576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:51.331756 (+ 180us) server_negotiation.cc:407] Connection header received
0808 16:56:51.332655 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:51.332656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:51.332669 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:51.332720 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:51.333255 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.333756 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.334668 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.334861 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.334956 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:51.337064 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:51.337065 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:51.337066 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:51.337073 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:51.338583 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:51.339021 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:51.339022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:51.339023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:51.339085 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:51.339336 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:51.339337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:51.339338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:51.339451 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:51.339524 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:51.339798 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 16:56:51.339888 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 16:56:51.340229 13688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:51.331461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:34711 (local address 127.8.14.65:39183)
0808 16:56:51.331642 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:51.331676 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:51.332568 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:51.332772 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:51.332775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:51.333012 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:51.333185 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:51.333189 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.333828 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.333829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:51.334591 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:51.334592 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.334975 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.334976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:51.335093 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:51.335491 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:51.335498 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:51.336954 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:51.338669 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:51.338676 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:51.338679 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:51.338951 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:51.339158 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:51.339159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:51.339159 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:51.339265 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:51.339527 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:51.339534 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:51.339665 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 16:56:51.339763 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 16:56:51.339918 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":106,"threads_started":1}
I20260808 16:56:51.340657 13685 heartbeater.cc:347] Connected to a master server at 127.8.14.126:34711
I20260808 16:56:51.340749 13685 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:51.340945 13685 heartbeater.cc:511] Master 127.8.14.126:34711 requested a full tablet report, sending...
I20260808 16:56:51.341439 13419 ts_manager.cc:194] Registered new tserver with Master: 749fdd71128f4a3c929449596aef607c (127.8.14.65:37947)
I20260808 16:56:51.342034 13419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:39183
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 16:56:51.361406 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:34711
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:51.442643 13693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:51.442831 13693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:51.442854 13693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:51.444073 13693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:51.444118 13693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:51.444142 13693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:56:51.445714 13693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:34711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:51.445892 13693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:51.446076 13693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:51.450845 13701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:51.450876 13700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:51.450943 13705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:51.451007 13693 server_base.cc:1034] running on GCE node
I20260808 16:56:51.451287 13693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:51.451478 13693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:51.452610 13693 hybrid_clock.cc:648] HybridClock initialized: now 1786208211452609 us; error 42 us; skew 500 ppm
Aug 08 16:56:51 dist-test-slave-1nl6 krb5kdc[13347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208211, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:51.455165 13693 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:56:51.455578 13693 webserver.cc:460] Webserver started at http://127.8.14.66:41975/ using document root <none> and password file <none>
I20260808 16:56:51.455842 13693 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:51.455890 13693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:51.456040 13693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:51.457029 13693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "e78168330aba449ebebf18d3a090cc70"
format_stamp: "Formatted at 2026-08-08 16:56:51 on dist-test-slave-1nl6"
server_key: "c5ff9eb1ea3bdfa068013a162e3a3e06"
server_key_iv: "d0848e503da199123f070152d4c4d240"
server_key_version: "encryptionkey@0"
I20260808 16:56:51.457381 13693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "e78168330aba449ebebf18d3a090cc70"
format_stamp: "Formatted at 2026-08-08 16:56:51 on dist-test-slave-1nl6"
server_key: "c5ff9eb1ea3bdfa068013a162e3a3e06"
server_key_iv: "d0848e503da199123f070152d4c4d240"
server_key_version: "encryptionkey@0"
I20260808 16:56:51.458640 13693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:51.459408 13714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:51.459599 13693 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:51.459692 13693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "e78168330aba449ebebf18d3a090cc70"
format_stamp: "Formatted at 2026-08-08 16:56:51 on dist-test-slave-1nl6"
server_key: "c5ff9eb1ea3bdfa068013a162e3a3e06"
server_key_iv: "d0848e503da199123f070152d4c4d240"
server_key_version: "encryptionkey@0"
I20260808 16:56:51.459765 13693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:51.483187 13693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:51.485196 13693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:51.485373 13693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:51.485631 13693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:51.485968 13693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:51.486028 13693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:51.486065 13693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:51.486092 13693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:51.492162 13693 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:40377
I20260808 16:56:51.492193 13874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:40377 every 8 connection(s)
I20260808 16:56:51.492537 13693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:56:51.496393 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 13693
I20260808 16:56:51.496519 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:56:51.496687 8249 external_mini_cluster.cc:1428] Setting key efd5b49bc011f58a422b103c0410142c
Aug 08 16:56:51 dist-test-slave-1nl6 krb5kdc[13347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208211, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:51.503285 13481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:51.493261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:43863 (local address 127.8.14.126:34711)
0808 16:56:51.493331 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:51.493333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:51.493542 (+ 209us) server_negotiation.cc:407] Connection header received
0808 16:56:51.494477 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:51.494478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:51.494486 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:51.494547 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:51.495230 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.495714 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.497089 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.497361 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.497430 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:51.500092 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:51.500094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:51.500094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:51.500103 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:51.501596 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:51.502167 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:51.502168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:51.502168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:51.502219 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:51.502522 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:51.502522 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:51.502523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:51.502624 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:51.502707 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:51.503083 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 16:56:51.503172 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 16:56:51.503587 13878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:51.493237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:34711 (local address 127.8.14.66:43863)
0808 16:56:51.493413 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:51.493455 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:51.494388 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:51.494605 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:51.494608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:51.494864 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:51.495151 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:51.495156 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.495795 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.495795 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:51.496965 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:51.496968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.497468 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.497469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:51.497630 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:51.498189 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:51.498200 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:51.499997 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:51.501685 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:51.501691 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:51.501694 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:51.502072 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:51.502281 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:51.502281 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:51.502282 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:51.502449 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:51.502718 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:51.502725 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:51.502923 (+ 198us) client_negotiation.cc:769] Sending connection context
0808 16:56:51.503054 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 16:56:51.503267 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
I20260808 16:56:51.504042 13876 heartbeater.cc:347] Connected to a master server at 127.8.14.126:34711
I20260808 16:56:51.504156 13876 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:51.504356 13876 heartbeater.cc:511] Master 127.8.14.126:34711 requested a full tablet report, sending...
I20260808 16:56:51.504742 13419 ts_manager.cc:194] Registered new tserver with Master: e78168330aba449ebebf18d3a090cc70 (127.8.14.66:40377)
I20260808 16:56:51.505116 13419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:43863
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 16:56:51.523173 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:34711
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:51.604921 13882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:51.605152 13882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:51.605192 13882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:51.606336 13882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:51.606405 13882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:51.606447 13882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:56:51.607980 13882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:34711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:51.608254 13882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:51.608489 13882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:51.613476 13889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:51.613492 13882 server_base.cc:1034] running on GCE node
W20260808 16:56:51.613477 13892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:51.613518 13887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:51.613878 13882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:51.614130 13882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:51.615295 13882 hybrid_clock.cc:648] HybridClock initialized: now 1786208211615286 us; error 37 us; skew 500 ppm
Aug 08 16:56:51 dist-test-slave-1nl6 krb5kdc[13347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208211, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:51.617748 13882 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:56:51.618245 13882 webserver.cc:460] Webserver started at http://127.8.14.67:40847/ using document root <none> and password file <none>
I20260808 16:56:51.618486 13882 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:51.618554 13882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:51.618706 13882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:51.619772 13882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "46b9f5b7592244c3bf2f12ac79d7d75c"
format_stamp: "Formatted at 2026-08-08 16:56:51 on dist-test-slave-1nl6"
server_key: "90ab70da1389e81bbe0d88779c7cfcb6"
server_key_iv: "6b8a037e75fe860726b29a5a885ef172"
server_key_version: "encryptionkey@0"
I20260808 16:56:51.620138 13882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "46b9f5b7592244c3bf2f12ac79d7d75c"
format_stamp: "Formatted at 2026-08-08 16:56:51 on dist-test-slave-1nl6"
server_key: "90ab70da1389e81bbe0d88779c7cfcb6"
server_key_iv: "6b8a037e75fe860726b29a5a885ef172"
server_key_version: "encryptionkey@0"
I20260808 16:56:51.621434 13882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:56:51.622113 13901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:51.622290 13882 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:51.622395 13882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "46b9f5b7592244c3bf2f12ac79d7d75c"
format_stamp: "Formatted at 2026-08-08 16:56:51 on dist-test-slave-1nl6"
server_key: "90ab70da1389e81bbe0d88779c7cfcb6"
server_key_iv: "6b8a037e75fe860726b29a5a885ef172"
server_key_version: "encryptionkey@0"
I20260808 16:56:51.622471 13882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:51.643442 13882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:51.645943 13882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:51.646126 13882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:51.646385 13882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:51.646720 13882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:51.646778 13882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:51.646842 13882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:51.646871 13882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:51.653187 13882 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:33027
I20260808 16:56:51.653231 14070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:33027 every 8 connection(s)
I20260808 16:56:51.653625 13882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 16:56:51.658819 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 13882
I20260808 16:56:51.658974 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:56:51.659216 8249 external_mini_cluster.cc:1428] Setting key ba815af039a3c2319427a25db656d69c
Aug 08 16:56:51 dist-test-slave-1nl6 krb5kdc[13347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208211, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:51.664417 13481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:51.654429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:50113 (local address 127.8.14.126:34711)
0808 16:56:51.654497 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:51.654499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:51.654711 (+ 212us) server_negotiation.cc:407] Connection header received
0808 16:56:51.655783 (+ 1072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:51.655784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:51.655798 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:51.655848 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:51.656504 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.657022 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.658073 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.658284 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.658374 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:51.661154 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:51.661156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:51.661157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:51.661165 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:51.662779 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:51.663257 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:51.663259 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:51.663260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:51.663355 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:51.663641 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:51.663642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:51.663642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:51.663771 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:51.663859 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:51.664192 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 16:56:51.664286 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 16:56:51.664577 14075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:51.654413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:34711 (local address 127.8.14.67:50113)
0808 16:56:51.654585 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:51.654622 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:51.655661 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:51.655923 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:51.655931 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:51.656195 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:51.656398 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:51.656402 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.657126 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.657127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:51.657960 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:51.657963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.658383 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.658384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:51.658527 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:51.659148 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:51.659157 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:51.661045 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:51.662881 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:51.662889 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:51.662892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:51.663174 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:51.663421 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:51.663422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:51.663422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:51.663564 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:51.663849 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:51.663857 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:51.664044 (+ 187us) client_negotiation.cc:769] Sending connection context
0808 16:56:51.664152 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 16:56:51.664330 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":78,"threads_started":1}
I20260808 16:56:51.665069 14073 heartbeater.cc:347] Connected to a master server at 127.8.14.126:34711
I20260808 16:56:51.665175 14073 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:51.665340 14073 heartbeater.cc:511] Master 127.8.14.126:34711 requested a full tablet report, sending...
I20260808 16:56:51.665743 13419 ts_manager.cc:194] Registered new tserver with Master: 46b9f5b7592244c3bf2f12ac79d7d75c (127.8.14.67:33027)
I20260808 16:56:51.666145 13419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:50113
I20260808 16:56:51.671172 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 16:56:51 dist-test-slave-1nl6 krb5kdc[13347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 16:56:51 dist-test-slave-1nl6 krb5kdc[13347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.65@KRBTEST.COM
I20260808 16:56:51.696123 14083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:51.688397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41512 (local address 127.8.14.65:37947)
0808 16:56:51.688584 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:51.688594 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:51.688604 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:56:51.688630 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:51.688632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:51.688646 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:51.688725 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:51.689352 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.690085 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.690854 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.691260 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.691353 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:51.693620 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:51.693627 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:51.693628 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:51.693635 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:51.695109 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:51.695438 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:51.695441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:51.695441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:51.695510 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:51.695670 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:51.695670 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:51.695671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:51.695814 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:51.695940 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:51.695979 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 16:56:51.696042 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
Aug 08 16:56:51 dist-test-slave-1nl6 krb5kdc[13347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:51.704787 13481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:51.697511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51288 (local address 127.8.14.126:34711)
0808 16:56:51.697570 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:51.697572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:51.697583 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:56:51.697646 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:51.697647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:51.697658 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:51.697728 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:51.698223 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.698715 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.699486 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:51.699679 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:51.699774 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:51.702002 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:51.702003 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:51.702004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:51.702012 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:51.703534 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:51.703920 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:51.703922 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:51.703923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:51.703987 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:51.704198 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:51.704199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:51.704199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:51.704373 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:51.704482 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:51.704594 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 16:56:51.704679 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 16:56:51.705051 13419 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51288
I20260808 16:56:51.705440 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 13505
I20260808 16:56:51.721292 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 13693
I20260808 16:56:51.733831 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 13882
I20260808 16:56:51.742594 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 13371
2026-08-08T16:56:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2023 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:51 dist-test-slave-1nl6 krb5kdc[14088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:51 dist-test-slave-1nl6 krb5kdc[14088](info): set up 2 sockets
Aug 08 16:56:51 dist-test-slave-1nl6 krb5kdc[14088](info): commencing operation
krb5kdc: starting...
W20260808 16:56:52.958570 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:56:53 dist-test-slave-1nl6 krb5kdc[14088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:56:53.075021 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:35653
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:35653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:53.157158 14113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:53.157379 14113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:53.157419 14113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:53.158587 14113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:53.158650 14113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:53.158676 14113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:53.158697 14113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:53.158725 14113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:53.160317 14113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:35653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:35653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:53.160576 14113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:53.160808 14113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:53.165584 14123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:53.165596 14121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:53.165593 14120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:53.165867 14113 server_base.cc:1034] running on GCE node
I20260808 16:56:53.166085 14113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:53.166363 14113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:53.167503 14113 hybrid_clock.cc:648] HybridClock initialized: now 1786208213167481 us; error 37 us; skew 500 ppm
Aug 08 16:56:53 dist-test-slave-1nl6 krb5kdc[14088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208213, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:53.170006 14113 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:56:53.170293 14113 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:53.170339 14113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:53.170429 14113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:53.171491 14113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "fb2b59376b424ec5afb72a4fd6f33aee"
format_stamp: "Formatted at 2026-08-08 16:56:53 on dist-test-slave-1nl6"
server_key: "83939a092dcb57d940c86f7381c057c9"
server_key_iv: "0ca38d2364312adee6433d96ad2d38d3"
server_key_version: "encryptionkey@0"
I20260808 16:56:53.171800 14113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "fb2b59376b424ec5afb72a4fd6f33aee"
format_stamp: "Formatted at 2026-08-08 16:56:53 on dist-test-slave-1nl6"
server_key: "83939a092dcb57d940c86f7381c057c9"
server_key_iv: "0ca38d2364312adee6433d96ad2d38d3"
server_key_version: "encryptionkey@0"
I20260808 16:56:53.173112 14113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:56:53.173774 14132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:53.173977 14113 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:53.174072 14113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "fb2b59376b424ec5afb72a4fd6f33aee"
format_stamp: "Formatted at 2026-08-08 16:56:53 on dist-test-slave-1nl6"
server_key: "83939a092dcb57d940c86f7381c057c9"
server_key_iv: "0ca38d2364312adee6433d96ad2d38d3"
server_key_version: "encryptionkey@0"
I20260808 16:56:53.174127 14113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:53.186419 14113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:53.188735 14113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:53.188848 14113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:53.192687 14113 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:35653
I20260808 16:56:53.192768 14210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:35653 every 8 connection(s)
I20260808 16:56:53.193044 14113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:53.194146 14213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:53.196484 14213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee: Bootstrap starting.
I20260808 16:56:53.197104 14213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:53.197417 14213 log.cc:824] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:53.198112 14213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee: No bootstrap required, opened a new log
I20260808 16:56:53.199687 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 14113
I20260808 16:56:53.199831 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:56:53.200008 8249 external_mini_cluster.cc:1428] Setting key a9b9b02307e17df36ae24559abea7de3
I20260808 16:56:53.200809 14213 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2b59376b424ec5afb72a4fd6f33aee" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35653 } }
I20260808 16:56:53.200985 14213 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:53.201022 14213 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b59376b424ec5afb72a4fd6f33aee, State: Initialized, Role: FOLLOWER
I20260808 16:56:53.201176 14213 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2b59376b424ec5afb72a4fd6f33aee" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35653 } }
I20260808 16:56:53.201292 14213 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:53.201333 14213 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:53.201401 14213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:53.202095 14213 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2b59376b424ec5afb72a4fd6f33aee" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35653 } }
I20260808 16:56:53.202204 14213 leader_election.cc:302] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb2b59376b424ec5afb72a4fd6f33aee; no voters:
I20260808 16:56:53.202370 14213 leader_election.cc:288] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:53.202438 14218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:53.202616 14213 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:53.202620 14218 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [term 1 LEADER]: Becoming Leader. State: Replica: fb2b59376b424ec5afb72a4fd6f33aee, State: Running, Role: LEADER
I20260808 16:56:53.202740 14218 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2b59376b424ec5afb72a4fd6f33aee" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35653 } }
I20260808 16:56:53.203080 14218 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb2b59376b424ec5afb72a4fd6f33aee. Latest consensus state: current_term: 1 leader_uuid: "fb2b59376b424ec5afb72a4fd6f33aee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2b59376b424ec5afb72a4fd6f33aee" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35653 } } }
I20260808 16:56:53.203164 14218 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:53.203331 14220 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb2b59376b424ec5afb72a4fd6f33aee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2b59376b424ec5afb72a4fd6f33aee" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35653 } } }
I20260808 16:56:53.203405 14220 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:53.203523 14229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:53.204217 14229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:53.205895 14229 catalog_manager.cc:1347] Generated new cluster ID: c9f5599d6c7c41e3901484a5901255e6
I20260808 16:56:53.205952 14229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:56:53 dist-test-slave-1nl6 krb5kdc[14088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:53.211700 14217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.200573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53390 (local address 127.8.14.126:35653)
0808 16:56:53.200787 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:53.200797 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:53.200831 (+ 34us) server_negotiation.cc:407] Connection header received
0808 16:56:53.200943 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:53.200950 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:53.201007 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:53.201155 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:53.201873 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.203370 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.204568 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.205086 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.205188 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.208086 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:53.208090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:53.208090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:53.208102 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:53.210026 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:53.210490 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:53.210493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:53.210493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:53.210605 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:53.210831 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:53.210831 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:53.210832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:53.211122 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:53.211276 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:53.211391 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 16:56:53.211514 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":91,"threads_started":1}
I20260808 16:56:53.217445 14229 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:53.217947 14229 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:53.223227 14229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee: Generated new TSK 0
I20260808 16:56:53.223399 14229 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 16:56:53.249594 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35653
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:53.331142 14243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:53.331339 14243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:53.331363 14243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:53.332499 14243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:53.332542 14243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:53.332594 14243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:56:53.334081 14243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:53.334316 14243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:53.334559 14243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:53.339519 14257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:53.339535 14251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:53.339564 14243 server_base.cc:1034] running on GCE node
W20260808 16:56:53.339534 14255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:53.339947 14243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:53.340142 14243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:53.341307 14243 hybrid_clock.cc:648] HybridClock initialized: now 1786208213341292 us; error 46 us; skew 500 ppm
Aug 08 16:56:53 dist-test-slave-1nl6 krb5kdc[14088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208213, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:53.343899 14243 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:56:53.344296 14243 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:53.344369 14243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:53.344523 14243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:53.345577 14243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "6ad086da18b348bba9ef552aa126b90d"
format_stamp: "Formatted at 2026-08-08 16:56:53 on dist-test-slave-1nl6"
server_key: "a023c140838f1b9faaa0030c7d356562"
server_key_iv: "53f7019f3834ecd0238cb01214d3c803"
server_key_version: "encryptionkey@0"
I20260808 16:56:53.345948 14243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "6ad086da18b348bba9ef552aa126b90d"
format_stamp: "Formatted at 2026-08-08 16:56:53 on dist-test-slave-1nl6"
server_key: "a023c140838f1b9faaa0030c7d356562"
server_key_iv: "53f7019f3834ecd0238cb01214d3c803"
server_key_version: "encryptionkey@0"
I20260808 16:56:53.347442 14243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 16:56:53.348146 14264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:53.348332 14243 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:53.348438 14243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "6ad086da18b348bba9ef552aa126b90d"
format_stamp: "Formatted at 2026-08-08 16:56:53 on dist-test-slave-1nl6"
server_key: "a023c140838f1b9faaa0030c7d356562"
server_key_iv: "53f7019f3834ecd0238cb01214d3c803"
server_key_version: "encryptionkey@0"
I20260808 16:56:53.348511 14243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:53.366067 14243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:53.368443 14243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:53.368602 14243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:53.368834 14243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:53.369159 14243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:53.369232 14243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:53.369274 14243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:53.369303 14243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:53.375111 14243 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:40139
I20260808 16:56:53.375161 14438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:40139 every 8 connection(s)
I20260808 16:56:53.375418 14243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
Aug 08 16:56:53 dist-test-slave-1nl6 krb5kdc[14088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208213, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:53.384191 14217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.376060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:44305 (local address 127.8.14.126:35653)
0808 16:56:53.376138 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:53.376141 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:53.376344 (+ 203us) server_negotiation.cc:407] Connection header received
0808 16:56:53.376870 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:53.376871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:53.376885 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:53.376938 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:53.377937 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.378506 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.379520 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.379708 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.379759 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.381818 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:53.381819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:53.381819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:53.381826 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:53.383058 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:53.383423 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:53.383424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:53.383424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:53.383473 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:53.383670 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:53.383671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:53.383671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:53.383764 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:53.383819 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:53.384062 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 16:56:53.384130 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 16:56:53.384346 14441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.376015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:35653 (local address 127.8.14.65:44305)
0808 16:56:53.376189 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:53.376240 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:53.376781 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:53.377013 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:53.377016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:53.377326 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:53.377860 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:53.377866 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.378582 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.378583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:53.379442 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:53.379445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.379779 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.379780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:53.379873 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.380367 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:53.380375 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:53.381718 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:53.383139 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:53.383145 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:53.383147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:53.383360 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:53.383528 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:53.383528 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:53.383529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:53.383621 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:53.383830 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:53.383834 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:53.383938 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 16:56:53.384036 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 16:56:53.384149 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
I20260808 16:56:53.384888 14439 heartbeater.cc:347] Connected to a master server at 127.8.14.126:35653
I20260808 16:56:53.384905 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 14243
I20260808 16:56:53.384980 14439 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:53.385025 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:56:53.385146 14439 heartbeater.cc:511] Master 127.8.14.126:35653 requested a full tablet report, sending...
I20260808 16:56:53.385241 8249 external_mini_cluster.cc:1428] Setting key 8a09eb6aa9a531b5808a2926571f4f48
I20260808 16:56:53.385666 14155 ts_manager.cc:194] Registered new tserver with Master: 6ad086da18b348bba9ef552aa126b90d (127.8.14.65:40139)
I20260808 16:56:53.386312 14155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:44305
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 16:56:53.408861 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35653
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:53.489090 14447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:53.489288 14447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:53.489311 14447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:53.490439 14447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:53.490480 14447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:53.490506 14447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:56:53.492053 14447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:53.492321 14447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:53.492550 14447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:53.497501 14462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:53.497567 14458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:53.497507 14455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:53.497727 14447 server_base.cc:1034] running on GCE node
I20260808 16:56:53.497942 14447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:53.498191 14447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:53.499333 14447 hybrid_clock.cc:648] HybridClock initialized: now 1786208213499324 us; error 33 us; skew 500 ppm
Aug 08 16:56:53 dist-test-slave-1nl6 krb5kdc[14088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208213, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:53.501822 14447 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:56:53.502174 14447 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:53.502223 14447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:53.502316 14447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:53.503427 14447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "24470b8658f34204996b264325306ee7"
format_stamp: "Formatted at 2026-08-08 16:56:53 on dist-test-slave-1nl6"
server_key: "860f776dd271e461559e8129a66b2b91"
server_key_iv: "a15470269a206203a14d82cf1c65bb68"
server_key_version: "encryptionkey@0"
I20260808 16:56:53.503723 14447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "24470b8658f34204996b264325306ee7"
format_stamp: "Formatted at 2026-08-08 16:56:53 on dist-test-slave-1nl6"
server_key: "860f776dd271e461559e8129a66b2b91"
server_key_iv: "a15470269a206203a14d82cf1c65bb68"
server_key_version: "encryptionkey@0"
I20260808 16:56:53.505013 14447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:56:53.505729 14471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:53.505896 14447 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:53.505997 14447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "24470b8658f34204996b264325306ee7"
format_stamp: "Formatted at 2026-08-08 16:56:53 on dist-test-slave-1nl6"
server_key: "860f776dd271e461559e8129a66b2b91"
server_key_iv: "a15470269a206203a14d82cf1c65bb68"
server_key_version: "encryptionkey@0"
I20260808 16:56:53.506068 14447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:53.517982 14447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:53.519816 14447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:53.519970 14447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:53.520185 14447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:53.520498 14447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:53.520550 14447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:53.520582 14447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:53.520609 14447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:53.526551 14447 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:34965
I20260808 16:56:53.526594 14652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:34965 every 8 connection(s)
I20260808 16:56:53.526844 14447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
Aug 08 16:56:53 dist-test-slave-1nl6 krb5kdc[14088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208213, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:53.533514 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 14447
I20260808 16:56:53.533629 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:56:53.533857 8249 external_mini_cluster.cc:1428] Setting key ac255d47f85bce4b7fb4ab038c4101bb
I20260808 16:56:53.535916 14217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.527542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:36309 (local address 127.8.14.126:35653)
0808 16:56:53.527629 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:53.527631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:53.527789 (+ 158us) server_negotiation.cc:407] Connection header received
0808 16:56:53.528223 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:53.528224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:53.528235 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:53.528291 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:53.529170 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.529675 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.530606 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.530818 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.530867 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.532984 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:53.532985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:53.532986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:53.532992 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:53.534467 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:53.534879 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:53.534880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:53.534880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:53.534986 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:53.535240 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:53.535241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:53.535242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:53.535359 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:53.535442 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:53.535704 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 16:56:53.535794 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 16:56:53.536170 14657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.527508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:35653 (local address 127.8.14.66:36309)
0808 16:56:53.527667 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:53.527701 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:53.528137 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:53.528347 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:53.528350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:53.528597 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:53.529093 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:53.529097 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.529752 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.529752 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:53.530535 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:53.530537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.530902 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.530903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:53.531072 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.531581 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:53.531589 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:53.532899 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:53.534548 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:53.534556 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:53.534558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:53.534784 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:53.535052 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:53.535053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:53.535053 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:53.535160 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:53.535446 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:53.535452 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:53.535573 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 16:56:53.535671 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 16:56:53.535818 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260808 16:56:53.536791 14654 heartbeater.cc:347] Connected to a master server at 127.8.14.126:35653
I20260808 16:56:53.536901 14654 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:53.537042 14654 heartbeater.cc:511] Master 127.8.14.126:35653 requested a full tablet report, sending...
I20260808 16:56:53.537420 14155 ts_manager.cc:194] Registered new tserver with Master: 24470b8658f34204996b264325306ee7 (127.8.14.66:34965)
I20260808 16:56:53.537930 14155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:36309
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 16:56:53.558682 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35653
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:36071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:53.639427 14662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:53.639609 14662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:53.639631 14662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:53.640659 14662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:53.640690 14662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:53.640715 14662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:56:53.642127 14662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:36071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:53.642302 14662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:53.642464 14662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:53.647325 14674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:53.647325 14672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:53.647360 14671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:53.647518 14662 server_base.cc:1034] running on GCE node
I20260808 16:56:53.647750 14662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:53.647981 14662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:53.649122 14662 hybrid_clock.cc:648] HybridClock initialized: now 1786208213649105 us; error 33 us; skew 500 ppm
Aug 08 16:56:53 dist-test-slave-1nl6 krb5kdc[14088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208213, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:53.651607 14662 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:56:53.651957 14662 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:53.652006 14662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:53.652096 14662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:53.653177 14662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "f193b9c99f974426a69f6511192c1a4e"
format_stamp: "Formatted at 2026-08-08 16:56:53 on dist-test-slave-1nl6"
server_key: "5c0e5ecef69ccfdce1902fd1176fd7dc"
server_key_iv: "f2a2def2b9496609bba5b6335098b67d"
server_key_version: "encryptionkey@0"
I20260808 16:56:53.653478 14662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "f193b9c99f974426a69f6511192c1a4e"
format_stamp: "Formatted at 2026-08-08 16:56:53 on dist-test-slave-1nl6"
server_key: "5c0e5ecef69ccfdce1902fd1176fd7dc"
server_key_iv: "f2a2def2b9496609bba5b6335098b67d"
server_key_version: "encryptionkey@0"
I20260808 16:56:53.654664 14662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 16:56:53.655398 14682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:53.655563 14662 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:53.655673 14662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "f193b9c99f974426a69f6511192c1a4e"
format_stamp: "Formatted at 2026-08-08 16:56:53 on dist-test-slave-1nl6"
server_key: "5c0e5ecef69ccfdce1902fd1176fd7dc"
server_key_iv: "f2a2def2b9496609bba5b6335098b67d"
server_key_version: "encryptionkey@0"
I20260808 16:56:53.655748 14662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:53.691841 14662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:53.693907 14662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:53.694077 14662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:53.694322 14662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:53.694639 14662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:53.694693 14662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:53.694734 14662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:53.694761 14662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:53.700822 14662 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:44445
I20260808 16:56:53.700870 14837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:44445 every 8 connection(s)
I20260808 16:56:53.701130 14662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 16:56:53.703720 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 14662
I20260808 16:56:53.703842 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:56:53.704111 8249 external_mini_cluster.cc:1428] Setting key 762474e4dcb6e5f6cbba05fb3d45fdf6
Aug 08 16:56:53 dist-test-slave-1nl6 krb5kdc[14088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208213, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:53.711141 14217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.701739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:42319 (local address 127.8.14.126:35653)
0808 16:56:53.701812 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:53.701814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:53.702074 (+ 260us) server_negotiation.cc:407] Connection header received
0808 16:56:53.702623 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:53.702624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:53.702635 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:53.702689 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:53.703985 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.704781 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.705924 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.706133 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.706209 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.708308 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:53.708309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:53.708310 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:53.708317 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:53.709977 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:53.710332 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:53.710333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:53.710333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:53.710386 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:53.710595 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:53.710595 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:53.710595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:53.710692 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:53.710743 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:53.711018 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 16:56:53.711079 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 16:56:53.711328 14840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.701739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:35653 (local address 127.8.14.67:42319)
0808 16:56:53.701905 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:53.701963 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:53.702535 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:53.702758 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:53.702761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:53.703080 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:53.703769 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:53.703777 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.704989 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.704989 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:53.705822 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:53.705825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.706217 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.706219 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:53.706344 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.706855 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:53.706863 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:53.708226 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:53.710048 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:53.710054 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:53.710055 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:53.710259 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:53.710444 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:53.710445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:53.710445 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:53.710541 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:53.710747 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:53.710751 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:53.710864 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 16:56:53.711002 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 16:56:53.711120 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260808 16:56:53.711827 14838 heartbeater.cc:347] Connected to a master server at 127.8.14.126:35653
I20260808 16:56:53.711942 14838 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:53.712090 14838 heartbeater.cc:511] Master 127.8.14.126:35653 requested a full tablet report, sending...
I20260808 16:56:53.712440 14155 ts_manager.cc:194] Registered new tserver with Master: f193b9c99f974426a69f6511192c1a4e (127.8.14.67:44445)
I20260808 16:56:53.712805 14155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:42319
I20260808 16:56:53.715728 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:56:53.722348 14217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.716985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53400 (local address 127.8.14.126:35653)
0808 16:56:53.717041 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:53.717043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:53.717151 (+ 108us) server_negotiation.cc:407] Connection header received
0808 16:56:53.717188 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:53.717188 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:53.717196 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:53.717241 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:53.717747 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.718186 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.718862 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.719091 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.719146 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.720168 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:53.720170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:53.720170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:53.720176 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:53.721507 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:53.721787 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:53.721788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:53.721788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:53.721838 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:53.721955 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:53.721955 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:53.721956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:53.722067 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:53.722143 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:53.722232 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 16:56:53.722290 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 16:56:53.724159 14155 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53400:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:56:53.725937 14155 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:56:53.732285 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.728587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:44445 (local address 127.0.0.1:51708)
0808 16:56:53.728697 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:53.728708 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:53.728759 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:53.729065 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:53.729066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:53.729069 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:53.729126 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:53.729128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.730056 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.730057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:53.731092 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:53.731095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.731963 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.731964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:53.732054 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.732068 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 16:56:53.732162 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 16:56:53.732215 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":69,"thread_start_us":71,"threads_started":1}
I20260808 16:56:53.732302 14861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.728773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51708 (local address 127.8.14.67:44445)
0808 16:56:53.728915 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:53.728919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:53.728929 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:56:53.728966 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:53.728967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:53.728978 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:53.729066 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:53.729249 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.729952 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.731236 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.731874 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.731950 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.732003 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:53.732186 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 16:56:53.732224 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260808 16:56:53.732723 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.728393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:40139 (local address 127.0.0.1:32796)
0808 16:56:53.728656 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:53.728667 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:53.728709 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:53.728983 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:53.728986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:53.728988 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:53.729060 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:53.729062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.730060 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.730061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:53.731580 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:53.731583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.732484 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.732485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:53.732546 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.732551 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:56:53.732611 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 16:56:53.732652 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":33,"thread_start_us":95,"threads_started":1}
I20260808 16:56:53.732872 14859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.728506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32796 (local address 127.8.14.65:40139)
0808 16:56:53.728813 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:53.728818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:53.728835 (+ 17us) server_negotiation.cc:407] Connection header received
0808 16:56:53.728866 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:53.728868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:53.728880 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:53.728969 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:53.729241 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.729959 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.731677 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.732332 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.732399 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.732452 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:53.732644 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 16:56:53.732692 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":72,"threads_started":1}
I20260808 16:56:53.733230 14747 tablet_service.cc:1467] Processing CreateTablet for tablet e7e3e0d3cfd441aa839551748cb6b3a3 (DEFAULT_TABLE table=test-table [id=5b53ec2187e6481d9b470d8da1f59dbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:53.733618 14747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7e3e0d3cfd441aa839551748cb6b3a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:53.733613 14342 tablet_service.cc:1467] Processing CreateTablet for tablet e7e3e0d3cfd441aa839551748cb6b3a3 (DEFAULT_TABLE table=test-table [id=5b53ec2187e6481d9b470d8da1f59dbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:53.733862 14855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.728383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:34965 (local address 127.0.0.1:45720)
0808 16:56:53.728564 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:53.728603 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:53.728675 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:53.730302 (+ 1627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:53.730303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:53.730305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:53.730342 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:53.730344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.731566 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.731567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:53.732652 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:53.732655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.733658 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.733659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:53.733722 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.733726 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:56:53.733759 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 16:56:53.733797 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":61,"threads_started":1}
I20260808 16:56:53.733960 14342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7e3e0d3cfd441aa839551748cb6b3a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:53.734184 14857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.728489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45720 (local address 127.8.14.66:34965)
0808 16:56:53.730142 (+ 1653us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:53.730146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:53.730155 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:56:53.730192 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:53.730194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:53.730209 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:53.730289 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:53.730407 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.731469 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.732746 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.733559 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.733974 (+ 415us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.734042 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:53.734070 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 16:56:53.734109 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1616,"thread_start_us":76,"threads_started":1}
I20260808 16:56:53.734794 14550 tablet_service.cc:1467] Processing CreateTablet for tablet e7e3e0d3cfd441aa839551748cb6b3a3 (DEFAULT_TABLE table=test-table [id=5b53ec2187e6481d9b470d8da1f59dbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:53.735158 14550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7e3e0d3cfd441aa839551748cb6b3a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:53.736678 14863 tablet_bootstrap.cc:492] T e7e3e0d3cfd441aa839551748cb6b3a3 P 6ad086da18b348bba9ef552aa126b90d: Bootstrap starting.
I20260808 16:56:53.737264 14863 tablet_bootstrap.cc:654] T e7e3e0d3cfd441aa839551748cb6b3a3 P 6ad086da18b348bba9ef552aa126b90d: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:53.737447 14870 tablet_bootstrap.cc:492] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e: Bootstrap starting.
I20260808 16:56:53.737825 14863 log.cc:824] T e7e3e0d3cfd441aa839551748cb6b3a3 P 6ad086da18b348bba9ef552aa126b90d: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:53.738180 14870 tablet_bootstrap.cc:654] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:53.738528 14872 tablet_bootstrap.cc:492] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7: Bootstrap starting.
I20260808 16:56:53.738669 14863 tablet_bootstrap.cc:492] T e7e3e0d3cfd441aa839551748cb6b3a3 P 6ad086da18b348bba9ef552aa126b90d: No bootstrap required, opened a new log
I20260808 16:56:53.738761 14863 ts_tablet_manager.cc:1397] T e7e3e0d3cfd441aa839551748cb6b3a3 P 6ad086da18b348bba9ef552aa126b90d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:56:53.738884 14870 log.cc:824] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:53.739322 14872 tablet_bootstrap.cc:654] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:53.739615 14870 tablet_bootstrap.cc:492] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e: No bootstrap required, opened a new log
I20260808 16:56:53.739698 14870 ts_tablet_manager.cc:1397] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 16:56:53.740000 14872 log.cc:824] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:53.740856 14872 tablet_bootstrap.cc:492] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7: No bootstrap required, opened a new log
I20260808 16:56:53.740940 14872 ts_tablet_manager.cc:1397] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 16:56:53.740979 14863 raft_consensus.cc:348] T e7e3e0d3cfd441aa839551748cb6b3a3 P 6ad086da18b348bba9ef552aa126b90d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24470b8658f34204996b264325306ee7" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34965 } } peers { permanent_uuid: "6ad086da18b348bba9ef552aa126b90d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40139 } } peers { permanent_uuid: "f193b9c99f974426a69f6511192c1a4e" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44445 } }
I20260808 16:56:53.741156 14863 raft_consensus.cc:374] T e7e3e0d3cfd441aa839551748cb6b3a3 P 6ad086da18b348bba9ef552aa126b90d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:53.741201 14863 raft_consensus.cc:729] T e7e3e0d3cfd441aa839551748cb6b3a3 P 6ad086da18b348bba9ef552aa126b90d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad086da18b348bba9ef552aa126b90d, State: Initialized, Role: FOLLOWER
I20260808 16:56:53.741344 14863 consensus_queue.cc:260] T e7e3e0d3cfd441aa839551748cb6b3a3 P 6ad086da18b348bba9ef552aa126b90d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24470b8658f34204996b264325306ee7" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34965 } } peers { permanent_uuid: "6ad086da18b348bba9ef552aa126b90d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40139 } } peers { permanent_uuid: "f193b9c99f974426a69f6511192c1a4e" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44445 } }
I20260808 16:56:53.741477 14870 raft_consensus.cc:348] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24470b8658f34204996b264325306ee7" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34965 } } peers { permanent_uuid: "6ad086da18b348bba9ef552aa126b90d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40139 } } peers { permanent_uuid: "f193b9c99f974426a69f6511192c1a4e" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44445 } }
I20260808 16:56:53.741641 14870 raft_consensus.cc:374] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:53.741633 14439 heartbeater.cc:503] Master 127.8.14.126:35653 was elected leader, sending a full tablet report...
I20260808 16:56:53.741689 14870 raft_consensus.cc:729] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f193b9c99f974426a69f6511192c1a4e, State: Initialized, Role: FOLLOWER
I20260808 16:56:53.741811 14870 consensus_queue.cc:260] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24470b8658f34204996b264325306ee7" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34965 } } peers { permanent_uuid: "6ad086da18b348bba9ef552aa126b90d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40139 } } peers { permanent_uuid: "f193b9c99f974426a69f6511192c1a4e" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44445 } }
I20260808 16:56:53.741626 14863 ts_tablet_manager.cc:1428] T e7e3e0d3cfd441aa839551748cb6b3a3 P 6ad086da18b348bba9ef552aa126b90d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 16:56:53.742077 14870 ts_tablet_manager.cc:1428] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 16:56:53.742194 14838 heartbeater.cc:503] Master 127.8.14.126:35653 was elected leader, sending a full tablet report...
I20260808 16:56:53.743341 14872 raft_consensus.cc:348] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24470b8658f34204996b264325306ee7" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34965 } } peers { permanent_uuid: "6ad086da18b348bba9ef552aa126b90d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40139 } } peers { permanent_uuid: "f193b9c99f974426a69f6511192c1a4e" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44445 } }
I20260808 16:56:53.743462 14872 raft_consensus.cc:374] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:53.743484 14872 raft_consensus.cc:729] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24470b8658f34204996b264325306ee7, State: Initialized, Role: FOLLOWER
I20260808 16:56:53.743587 14872 consensus_queue.cc:260] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24470b8658f34204996b264325306ee7" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34965 } } peers { permanent_uuid: "6ad086da18b348bba9ef552aa126b90d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40139 } } peers { permanent_uuid: "f193b9c99f974426a69f6511192c1a4e" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44445 } }
I20260808 16:56:53.743808 14872 ts_tablet_manager.cc:1428] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 16:56:53.743870 14654 heartbeater.cc:503] Master 127.8.14.126:35653 was elected leader, sending a full tablet report...
W20260808 16:56:53.778260 14656 tablet.cc:2369] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:56:53.805300 14883 raft_consensus.cc:482] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:56:53.805428 14883 raft_consensus.cc:504] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24470b8658f34204996b264325306ee7" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34965 } } peers { permanent_uuid: "6ad086da18b348bba9ef552aa126b90d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40139 } } peers { permanent_uuid: "f193b9c99f974426a69f6511192c1a4e" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44445 } }
I20260808 16:56:53.806015 14883 leader_election.cc:288] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24470b8658f34204996b264325306ee7 (127.8.14.66:34965), 6ad086da18b348bba9ef552aa126b90d (127.8.14.65:40139)
I20260808 16:56:53.810317 14859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.806411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:53027 (local address 127.8.14.65:40139)
0808 16:56:53.806471 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:53.806474 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:53.806517 (+ 43us) server_negotiation.cc:407] Connection header received
0808 16:56:53.806612 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:53.806613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:53.806623 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:53.806673 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:53.806843 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.807580 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.809259 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.809876 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.809938 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.809947 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:53.810221 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 16:56:53.810262 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:56:53.810295 14857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.806260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:38253 (local address 127.8.14.66:34965)
0808 16:56:53.806311 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:53.806313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:53.806325 (+ 12us) server_negotiation.cc:407] Connection header received
0808 16:56:53.806457 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:53.806458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:53.806469 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:53.806526 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:56:53.806761 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.807588 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.809259 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.809840 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.809919 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.809933 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:53.810155 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 16:56:53.810231 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 16:56:53.810343 14840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.806201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:34965 (local address 127.8.14.67:38253)
0808 16:56:53.806261 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:53.806275 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:53.806330 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:53.806551 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:53.806552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:53.806554 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:53.806642 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:53.806648 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.807674 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.807675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:53.809148 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:53.809152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.809946 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.809947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:53.810044 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.810046 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:56:53.810115 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 16:56:53.810185 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":113}
I20260808 16:56:53.810354 14886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.806206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:40139 (local address 127.8.14.67:53027)
0808 16:56:53.806454 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:53.806466 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:53.806526 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:53.806700 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:53.806700 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:53.806702 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:56:53.806778 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:53.806779 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.807674 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.807675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:53.809148 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:53.809152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.809948 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.809949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:53.810042 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.810046 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:56:53.810173 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 16:56:53.810205 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":74,"thread_start_us":102,"threads_started":1}
I20260808 16:56:53.810686 14576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7e3e0d3cfd441aa839551748cb6b3a3" candidate_uuid: "f193b9c99f974426a69f6511192c1a4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24470b8658f34204996b264325306ee7" is_pre_election: true
I20260808 16:56:53.810868 14576 raft_consensus.cc:2454] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f193b9c99f974426a69f6511192c1a4e in term 0.
I20260808 16:56:53.811177 14691 leader_election.cc:302] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24470b8658f34204996b264325306ee7, f193b9c99f974426a69f6511192c1a4e; no voters:
I20260808 16:56:53.811169 14366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7e3e0d3cfd441aa839551748cb6b3a3" candidate_uuid: "f193b9c99f974426a69f6511192c1a4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad086da18b348bba9ef552aa126b90d" is_pre_election: true
I20260808 16:56:53.811321 14366 raft_consensus.cc:2454] T e7e3e0d3cfd441aa839551748cb6b3a3 P 6ad086da18b348bba9ef552aa126b90d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f193b9c99f974426a69f6511192c1a4e in term 0.
I20260808 16:56:53.811399 14883 raft_consensus.cc:2781] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:56:53.811474 14883 raft_consensus.cc:482] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:56:53.811504 14883 raft_consensus.cc:3037] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:53.812312 14883 raft_consensus.cc:504] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24470b8658f34204996b264325306ee7" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34965 } } peers { permanent_uuid: "6ad086da18b348bba9ef552aa126b90d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40139 } } peers { permanent_uuid: "f193b9c99f974426a69f6511192c1a4e" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44445 } }
I20260808 16:56:53.812547 14883 leader_election.cc:288] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [CANDIDATE]: Term 1 election: Requested vote from peers 24470b8658f34204996b264325306ee7 (127.8.14.66:34965), 6ad086da18b348bba9ef552aa126b90d (127.8.14.65:40139)
I20260808 16:56:53.812642 14366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7e3e0d3cfd441aa839551748cb6b3a3" candidate_uuid: "f193b9c99f974426a69f6511192c1a4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad086da18b348bba9ef552aa126b90d"
I20260808 16:56:53.812671 14576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7e3e0d3cfd441aa839551748cb6b3a3" candidate_uuid: "f193b9c99f974426a69f6511192c1a4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24470b8658f34204996b264325306ee7"
I20260808 16:56:53.812738 14576 raft_consensus.cc:3037] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:53.812734 14366 raft_consensus.cc:3037] T e7e3e0d3cfd441aa839551748cb6b3a3 P 6ad086da18b348bba9ef552aa126b90d [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:53.813731 14576 raft_consensus.cc:2454] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f193b9c99f974426a69f6511192c1a4e in term 1.
I20260808 16:56:53.813731 14366 raft_consensus.cc:2454] T e7e3e0d3cfd441aa839551748cb6b3a3 P 6ad086da18b348bba9ef552aa126b90d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f193b9c99f974426a69f6511192c1a4e in term 1.
I20260808 16:56:53.813932 14691 leader_election.cc:302] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24470b8658f34204996b264325306ee7, f193b9c99f974426a69f6511192c1a4e; no voters:
I20260808 16:56:53.814066 14883 raft_consensus.cc:2781] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:53.814237 14883 raft_consensus.cc:686] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [term 1 LEADER]: Becoming Leader. State: Replica: f193b9c99f974426a69f6511192c1a4e, State: Running, Role: LEADER
I20260808 16:56:53.814369 14883 consensus_queue.cc:237] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24470b8658f34204996b264325306ee7" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34965 } } peers { permanent_uuid: "6ad086da18b348bba9ef552aa126b90d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40139 } } peers { permanent_uuid: "f193b9c99f974426a69f6511192c1a4e" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44445 } }
I20260808 16:56:53.815289 14155 catalog_manager.cc:5495] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e reported cstate change: term changed from 0 to 1, leader changed from <none> to f193b9c99f974426a69f6511192c1a4e (127.8.14.67). New cstate: current_term: 1 leader_uuid: "f193b9c99f974426a69f6511192c1a4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24470b8658f34204996b264325306ee7" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad086da18b348bba9ef552aa126b90d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f193b9c99f974426a69f6511192c1a4e" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44445 } health_report { overall_health: HEALTHY } } }
I20260808 16:56:53.826241 14861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.823558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51718 (local address 127.8.14.67:44445)
0808 16:56:53.823617 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:53.823619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:53.823630 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:56:53.823693 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:53.823694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:53.823714 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:53.823810 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:56:53.823971 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.824487 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.825327 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:53.825556 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:53.825653 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:53.825855 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:56:53.825968 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:56:53.826062 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:53.826107 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 16:56:53.826142 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 16:56:53.828109 14576 raft_consensus.cc:1261] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7 [term 1 FOLLOWER]: Refusing update from remote peer f193b9c99f974426a69f6511192c1a4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:53.828109 14366 raft_consensus.cc:1261] T e7e3e0d3cfd441aa839551748cb6b3a3 P 6ad086da18b348bba9ef552aa126b90d [term 1 FOLLOWER]: Refusing update from remote peer f193b9c99f974426a69f6511192c1a4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:53.828457 14883 consensus_queue.cc:1035] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad086da18b348bba9ef552aa126b90d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:53.828621 14887 consensus_queue.cc:1035] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "24470b8658f34204996b264325306ee7" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:53.831593 14892 mvcc.cc:205] Tried to move back new op lower bound from 7316308843837169664 to 7316308843784466432. Current Snapshot: MvccSnapshot[applied={T|T < 7316308843837169664}]
I20260808 16:56:53.831977 14891 mvcc.cc:205] Tried to move back new op lower bound from 7316308843837169664 to 7316308843784466432. Current Snapshot: MvccSnapshot[applied={T|T < 7316308843837169664}]
I20260808 16:56:53.833153 14155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:56:53.833259 14155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:56:53.833993 14155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb2b59376b424ec5afb72a4fd6f33aee: Sending DeleteTablet for 3 replicas of tablet e7e3e0d3cfd441aa839551748cb6b3a3
I20260808 16:56:53.834316 14550 tablet_service.cc:1514] Processing DeleteTablet for tablet e7e3e0d3cfd441aa839551748cb6b3a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:53 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:45720
I20260808 16:56:53.834332 14342 tablet_service.cc:1514] Processing DeleteTablet for tablet e7e3e0d3cfd441aa839551748cb6b3a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:53 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:32796
I20260808 16:56:53.834343 14747 tablet_service.cc:1514] Processing DeleteTablet for tablet e7e3e0d3cfd441aa839551748cb6b3a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:53 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:51708
I20260808 16:56:53.834611 14906 tablet_replica.cc:330] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7: stopping tablet replica
I20260808 16:56:53.834709 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 14243
I20260808 16:56:53.838450 14906 raft_consensus.cc:2229] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:56:53.838562 14906 raft_consensus.cc:2258] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:53.839583 14908 tablet_replica.cc:330] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e: stopping tablet replica
I20260808 16:56:53.839679 14908 raft_consensus.cc:2229] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:56:53.839777 14908 raft_consensus.cc:2258] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:53.840654 14906 ts_tablet_manager.cc:1905] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:53.842960 14906 ts_tablet_manager.cc:1918] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:56:53.843032 14906 log.cc:1196] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/e7e3e0d3cfd441aa839551748cb6b3a3
I20260808 16:56:53.843320 14906 ts_tablet_manager.cc:1939] T e7e3e0d3cfd441aa839551748cb6b3a3 P 24470b8658f34204996b264325306ee7: Deleting consensus metadata
I20260808 16:56:53.847760 14908 ts_tablet_manager.cc:1905] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:53.849349 14908 ts_tablet_manager.cc:1918] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:56:53.849431 14908 log.cc:1196] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/e7e3e0d3cfd441aa839551748cb6b3a3
I20260808 16:56:53.849653 14138 catalog_manager.cc:4841] TS 24470b8658f34204996b264325306ee7 (127.8.14.66:34965): tablet e7e3e0d3cfd441aa839551748cb6b3a3 (table test-table [id=5b53ec2187e6481d9b470d8da1f59dbf]) successfully deleted
I20260808 16:56:53.849728 14908 ts_tablet_manager.cc:1939] T e7e3e0d3cfd441aa839551748cb6b3a3 P f193b9c99f974426a69f6511192c1a4e: Deleting consensus metadata
I20260808 16:56:53.850174 14138 catalog_manager.cc:4841] TS f193b9c99f974426a69f6511192c1a4e (127.8.14.67:44445): tablet e7e3e0d3cfd441aa839551748cb6b3a3 (table test-table [id=5b53ec2187e6481d9b470d8da1f59dbf]) successfully deleted
W20260808 16:56:53.855664 14133 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.14.65:40139 (error 108)
I20260808 16:56:53.855917 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 14447
I20260808 16:56:53.856550 14855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:53.856145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:40139 (local address 127.0.0.1:32806)
0808 16:56:53.856470 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:53.856511 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.65:40139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303}
W20260808 16:56:53.857676 14133 catalog_manager.cc:4584] TS 6ad086da18b348bba9ef552aa126b90d (127.8.14.65:40139): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e7e3e0d3cfd441aa839551748cb6b3a3: Network error: Client connection negotiation failed: client connection to 127.8.14.65:40139: connect: Connection refused (error 111)
I20260808 16:56:53.870306 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 14662
I20260808 16:56:53.885461 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 14113
2026-08-08T16:56:53Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2141 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T16:56:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:53Z Disabled control of system clock
I20260808 16:56:53.917035 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:43045
--webserver_interface=127.8.14.126
--webserver_port=0
--builtin_ntp_servers=127.8.14.84:33303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:43045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 16:56:53.998688 14919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:53.998884 14919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:53.998991 14919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:54.000057 14919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:54.000090 14919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:54.000103 14919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:54.000115 14919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:54.000128 14919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:54.001463 14919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:33303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:43045
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:43045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:54.001652 14919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:54.001875 14919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:54.006755 14930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:54.006743 14925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:54.006760 14926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:54.007067 14919 server_base.cc:1034] running on GCE node
I20260808 16:56:54.007375 14919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:54.007702 14919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:54.008889 14919 hybrid_clock.cc:648] HybridClock initialized: now 1786208214008864 us; error 36 us; skew 500 ppm
I20260808 16:56:54.010329 14919 webserver.cc:460] Webserver started at http://127.8.14.126:46173/ using document root <none> and password file <none>
I20260808 16:56:54.010593 14919 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:54.010674 14919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:54.010864 14919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:54.012038 14919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "18393d8b9f3f4ecd84483ddff4966ee7"
format_stamp: "Formatted at 2026-08-08 16:56:54 on dist-test-slave-1nl6"
server_key: "d8aa48d328fd18e96893928812f8868c"
server_key_iv: "e5b8615e79e4e3edfe854d1cfc79a966"
server_key_version: "encryptionkey@0"
I20260808 16:56:54.012461 14919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "18393d8b9f3f4ecd84483ddff4966ee7"
format_stamp: "Formatted at 2026-08-08 16:56:54 on dist-test-slave-1nl6"
server_key: "d8aa48d328fd18e96893928812f8868c"
server_key_iv: "e5b8615e79e4e3edfe854d1cfc79a966"
server_key_version: "encryptionkey@0"
I20260808 16:56:54.013864 14919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 16:56:54.014647 14939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:54.014835 14919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:54.014967 14919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "18393d8b9f3f4ecd84483ddff4966ee7"
format_stamp: "Formatted at 2026-08-08 16:56:54 on dist-test-slave-1nl6"
server_key: "d8aa48d328fd18e96893928812f8868c"
server_key_iv: "e5b8615e79e4e3edfe854d1cfc79a966"
server_key_version: "encryptionkey@0"
I20260808 16:56:54.015058 14919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:54.018616 14919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:54.018970 14919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:54.019142 14919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:54.023327 14919 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:43045
I20260808 16:56:54.023370 15028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:43045 every 8 connection(s)
I20260808 16:56:54.023747 14919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:54.024367 15029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:54.027117 15029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7: Bootstrap starting.
I20260808 16:56:54.027704 15029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:54.027993 15029 log.cc:824] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:54.028694 15029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7: No bootstrap required, opened a new log
I20260808 16:56:54.030200 15029 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18393d8b9f3f4ecd84483ddff4966ee7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 43045 } }
I20260808 16:56:54.030351 15029 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:54.030401 15029 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18393d8b9f3f4ecd84483ddff4966ee7, State: Initialized, Role: FOLLOWER
I20260808 16:56:54.030539 15029 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18393d8b9f3f4ecd84483ddff4966ee7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 43045 } }
I20260808 16:56:54.030619 15029 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:54.030669 15029 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:54.030718 15029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:54.031368 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 14919
I20260808 16:56:54.031417 15029 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18393d8b9f3f4ecd84483ddff4966ee7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 43045 } }
I20260808 16:56:54.031461 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:56:54.031515 15029 leader_election.cc:302] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18393d8b9f3f4ecd84483ddff4966ee7; no voters:
I20260808 16:56:54.031652 8249 external_mini_cluster.cc:1428] Setting key f28062f902d732c342b9b8a238d2aca6
I20260808 16:56:54.031692 15029 leader_election.cc:288] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:54.031745 15033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:54.031890 15033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [term 1 LEADER]: Becoming Leader. State: Replica: 18393d8b9f3f4ecd84483ddff4966ee7, State: Running, Role: LEADER
I20260808 16:56:54.031960 15029 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:54.031993 15033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18393d8b9f3f4ecd84483ddff4966ee7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 43045 } }
I20260808 16:56:54.032348 15033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18393d8b9f3f4ecd84483ddff4966ee7. Latest consensus state: current_term: 1 leader_uuid: "18393d8b9f3f4ecd84483ddff4966ee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18393d8b9f3f4ecd84483ddff4966ee7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 43045 } } }
I20260808 16:56:54.032436 15033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:54.032580 15035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18393d8b9f3f4ecd84483ddff4966ee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18393d8b9f3f4ecd84483ddff4966ee7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 43045 } } }
I20260808 16:56:54.032641 15035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:54.032665 15044 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:54.033921 15044 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:54.034577 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.032382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48310 (local address 127.8.14.126:43045)
0808 16:56:54.032824 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:54.032833 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:54.032870 (+ 37us) server_negotiation.cc:407] Connection header received
0808 16:56:54.032988 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:54.032995 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:54.033042 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:54.033189 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:54.033999 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:54.034002 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:54.034003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:54.034007 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:54.034051 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:54.034065 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:54.034336 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:54.034368 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 16:56:54.034425 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":73,"threads_started":1}
I20260808 16:56:54.035548 15069 mvcc.cc:205] Tried to move back new op lower bound from 7316308844686610432 to 7316308844675616768. Current Snapshot: MvccSnapshot[applied={T|T < 7316308844686610432}]
I20260808 16:56:54.035720 15044 catalog_manager.cc:1347] Generated new cluster ID: 1b27de904b1e4210847abb93971b0719
I20260808 16:56:54.035779 15044 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:54.048396 15044 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:54.048557 15044 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:54.065418 15044 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7: Generated new TSK 0
I20260808 16:56:54.065622 15044 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:54.073158 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:43045
--builtin_ntp_servers=127.8.14.84:33303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 16:56:54.157263 15071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:54.157455 15071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:54.157477 15071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:54.158599 15071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:54.158639 15071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:54.158671 15071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:56:54.160049 15071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:33303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:43045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:54.160291 15071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:54.160586 15071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:54.165422 15082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:54.165428 15079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:54.165504 15071 server_base.cc:1034] running on GCE node
W20260808 16:56:54.165428 15080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:54.165948 15071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:54.166154 15071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:54.167294 15071 hybrid_clock.cc:648] HybridClock initialized: now 1786208214167272 us; error 42 us; skew 500 ppm
I20260808 16:56:54.172509 15071 webserver.cc:460] Webserver started at http://127.8.14.65:39889/ using document root <none> and password file <none>
I20260808 16:56:54.172770 15071 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:54.172818 15071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:54.172976 15071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:54.173992 15071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "c74c34eb862a4712af375096109974d8"
format_stamp: "Formatted at 2026-08-08 16:56:54 on dist-test-slave-1nl6"
server_key: "467bc94cd288bf12993ccc827fe8464d"
server_key_iv: "2ee255c7e93e0d4e0f45a8b6d599e3e5"
server_key_version: "encryptionkey@0"
I20260808 16:56:54.174337 15071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "c74c34eb862a4712af375096109974d8"
format_stamp: "Formatted at 2026-08-08 16:56:54 on dist-test-slave-1nl6"
server_key: "467bc94cd288bf12993ccc827fe8464d"
server_key_iv: "2ee255c7e93e0d4e0f45a8b6d599e3e5"
server_key_version: "encryptionkey@0"
I20260808 16:56:54.175732 15071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:54.176523 15090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:54.176705 15071 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:54.176805 15071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "c74c34eb862a4712af375096109974d8"
format_stamp: "Formatted at 2026-08-08 16:56:54 on dist-test-slave-1nl6"
server_key: "467bc94cd288bf12993ccc827fe8464d"
server_key_iv: "2ee255c7e93e0d4e0f45a8b6d599e3e5"
server_key_version: "encryptionkey@0"
I20260808 16:56:54.176877 15071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:54.180305 15071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:54.180567 15071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:54.180668 15071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:54.180855 15071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:54.181154 15071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:54.181188 15071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:54.181207 15071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:54.181262 15071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:54.187036 15071 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:38085
I20260808 16:56:54.187110 15261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:38085 every 8 connection(s)
I20260808 16:56:54.187393 15071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:56:54.188572 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 15071
I20260808 16:56:54.188658 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:56:54.188848 8249 external_mini_cluster.cc:1428] Setting key 6c51e366f8a29538b316e6a855c26c67
I20260808 16:56:54.189745 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:43045
--builtin_ntp_servers=127.8.14.84:33303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 16:56:54.190667 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.188104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:38791 (local address 127.8.14.126:43045)
0808 16:56:54.188173 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:54.188175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:54.188409 (+ 234us) server_negotiation.cc:407] Connection header received
0808 16:56:54.189008 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:54.189009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:54.189017 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:54.189074 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:54.190376 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:54.190377 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:54.190378 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:54.190382 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:54.190399 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:54.190404 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:54.190480 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:54.190579 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 16:56:54.190596 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 16:56:54.190860 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.188093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:43045 (local address 127.8.14.65:38791)
0808 16:56:54.188292 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:54.188323 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:54.188914 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:54.189144 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:54.189148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:54.189157 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:54.190216 (+ 1059us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:56:54.190216 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:54.190247 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:56:54.190275 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:54.190482 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:54.190489 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:54.190492 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:56:54.190589 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 16:56:54.190650 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
I20260808 16:56:54.191641 15262 heartbeater.cc:347] Connected to a master server at 127.8.14.126:43045
I20260808 16:56:54.191746 15262 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:54.191895 15262 heartbeater.cc:511] Master 127.8.14.126:43045 requested a full tablet report, sending...
I20260808 16:56:54.192430 14967 ts_manager.cc:194] Registered new tserver with Master: c74c34eb862a4712af375096109974d8 (127.8.14.65:38085)
W20260808 16:56:54.275449 15265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:54.275668 15265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:54.275699 15265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:54.276888 15265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:54.276924 15265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:54.276952 15265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:56:54.278309 15265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:33303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:43045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:54.278496 15265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:54.278728 15265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:54.283852 15277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:54.283875 15273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:54.283867 15274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:54.283965 15265 server_base.cc:1034] running on GCE node
I20260808 16:56:54.284281 15265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:54.284534 15265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:54.285713 15265 hybrid_clock.cc:648] HybridClock initialized: now 1786208214285708 us; error 36 us; skew 500 ppm
I20260808 16:56:54.287132 15265 webserver.cc:460] Webserver started at http://127.8.14.66:45737/ using document root <none> and password file <none>
I20260808 16:56:54.287402 15265 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:54.287472 15265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:54.287609 15265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:54.288623 15265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "38649115c1634e938110e0c4d901d8e2"
format_stamp: "Formatted at 2026-08-08 16:56:54 on dist-test-slave-1nl6"
server_key: "aea08f17cd2eaae68d82a1e0d2693878"
server_key_iv: "9a5d0fc634f8872745c0b33815171f28"
server_key_version: "encryptionkey@0"
I20260808 16:56:54.288978 15265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "38649115c1634e938110e0c4d901d8e2"
format_stamp: "Formatted at 2026-08-08 16:56:54 on dist-test-slave-1nl6"
server_key: "aea08f17cd2eaae68d82a1e0d2693878"
server_key_iv: "9a5d0fc634f8872745c0b33815171f28"
server_key_version: "encryptionkey@0"
I20260808 16:56:54.290261 15265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:54.291006 15287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:54.291198 15265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:54.291301 15265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "38649115c1634e938110e0c4d901d8e2"
format_stamp: "Formatted at 2026-08-08 16:56:54 on dist-test-slave-1nl6"
server_key: "aea08f17cd2eaae68d82a1e0d2693878"
server_key_iv: "9a5d0fc634f8872745c0b33815171f28"
server_key_version: "encryptionkey@0"
I20260808 16:56:54.291383 15265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:54.294624 15265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:54.294875 15265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:54.295040 15265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:54.295250 15265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:54.295542 15265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:54.295590 15265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:54.295629 15265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:54.295656 15265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:54.301213 15265 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:40937
I20260808 16:56:54.301249 15448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:40937 every 8 connection(s)
I20260808 16:56:54.301565 15265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:56:54.304347 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.302228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:55109 (local address 127.8.14.126:43045)
0808 16:56:54.302275 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:54.302278 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:54.302480 (+ 202us) server_negotiation.cc:407] Connection header received
0808 16:56:54.302960 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:54.302961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:54.302972 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:54.303037 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:54.304107 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:54.304108 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:54.304133 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:54.304138 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:54.304155 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:54.304159 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:54.304210 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:54.304280 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 16:56:54.304292 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:56:54.304519 15452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.302195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:43045 (local address 127.8.14.66:55109)
0808 16:56:54.302371 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:54.302409 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:54.302835 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:54.303103 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:54.303106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:54.303114 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:54.303971 (+ 857us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:56:54.303971 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:54.303995 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:56:54.304021 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:54.304217 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:54.304222 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:54.304224 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:56:54.304286 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 16:56:54.304333 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260808 16:56:54.304984 15449 heartbeater.cc:347] Connected to a master server at 127.8.14.126:43045
I20260808 16:56:54.305107 15449 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:54.305238 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 15265
I20260808 16:56:54.305269 15449 heartbeater.cc:511] Master 127.8.14.126:43045 requested a full tablet report, sending...
I20260808 16:56:54.305367 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:56:54.305558 8249 external_mini_cluster.cc:1428] Setting key 848aa53de70480cca7a88bcaf8431252
I20260808 16:56:54.305650 14967 ts_manager.cc:194] Registered new tserver with Master: 38649115c1634e938110e0c4d901d8e2 (127.8.14.66:40937)
I20260808 16:56:54.306383 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:43045
--builtin_ntp_servers=127.8.14.84:33303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 16:56:54.389361 15453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:54.389564 15453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:54.389586 15453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:54.390762 15453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:54.390803 15453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:54.390829 15453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:56:54.392226 15453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:33303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:43045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:54.392493 15453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:54.392789 15453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:54.397887 15467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:54.397884 15464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:54.397884 15473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:54.398151 15453 server_base.cc:1034] running on GCE node
I20260808 16:56:54.398394 15453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:54.398588 15453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:54.399748 15453 hybrid_clock.cc:648] HybridClock initialized: now 1786208214399733 us; error 27 us; skew 500 ppm
I20260808 16:56:54.401098 15453 webserver.cc:460] Webserver started at http://127.8.14.67:40561/ using document root <none> and password file <none>
I20260808 16:56:54.401355 15453 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:54.401428 15453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:54.401580 15453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:54.402619 15453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "9825670756d84f25af5639804ca661dc"
format_stamp: "Formatted at 2026-08-08 16:56:54 on dist-test-slave-1nl6"
server_key: "cd3fc31a692411c9d150c8826b4d4bfe"
server_key_iv: "845a22cb5a2f154e65e118b79f2d204b"
server_key_version: "encryptionkey@0"
I20260808 16:56:54.403049 15453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "9825670756d84f25af5639804ca661dc"
format_stamp: "Formatted at 2026-08-08 16:56:54 on dist-test-slave-1nl6"
server_key: "cd3fc31a692411c9d150c8826b4d4bfe"
server_key_iv: "845a22cb5a2f154e65e118b79f2d204b"
server_key_version: "encryptionkey@0"
I20260808 16:56:54.404397 15453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:56:54.405126 15480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:54.405295 15453 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:56:54.405414 15453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "9825670756d84f25af5639804ca661dc"
format_stamp: "Formatted at 2026-08-08 16:56:54 on dist-test-slave-1nl6"
server_key: "cd3fc31a692411c9d150c8826b4d4bfe"
server_key_iv: "845a22cb5a2f154e65e118b79f2d204b"
server_key_version: "encryptionkey@0"
I20260808 16:56:54.405488 15453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:54.408810 15453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:54.409062 15453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:54.409206 15453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:54.409415 15453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:54.409718 15453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:54.409771 15453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:54.409817 15453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:54.409845 15453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:54.417497 15453 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:35191
I20260808 16:56:54.417554 15654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:35191 every 8 connection(s)
I20260808 16:56:54.417878 15453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 16:56:54.420648 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.418522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:53339 (local address 127.8.14.126:43045)
0808 16:56:54.418569 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:54.418571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:54.418837 (+ 266us) server_negotiation.cc:407] Connection header received
0808 16:56:54.419348 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:54.419349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:54.419354 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:54.419406 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:54.420468 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:54.420469 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:54.420470 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:54.420474 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:54.420488 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:54.420493 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:54.420541 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:54.420584 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 16:56:54.420596 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:56:54.420866 15659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.418518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:43045 (local address 127.8.14.67:53339)
0808 16:56:54.418694 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:54.418750 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:54.419252 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:54.419457 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:54.419460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:54.419468 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:54.420326 (+ 858us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:56:54.420326 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:54.420354 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:56:54.420380 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:54.420538 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:54.420540 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:54.420542 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:56:54.420591 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 16:56:54.420636 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260808 16:56:54.421402 15656 heartbeater.cc:347] Connected to a master server at 127.8.14.126:43045
I20260808 16:56:54.421509 15656 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:54.421597 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 15453
I20260808 16:56:54.421669 15656 heartbeater.cc:511] Master 127.8.14.126:43045 requested a full tablet report, sending...
I20260808 16:56:54.421711 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:56:54.421929 8249 external_mini_cluster.cc:1428] Setting key e715e930430e3be3fb7ae2a8416761d4
I20260808 16:56:54.422091 14967 ts_manager.cc:194] Registered new tserver with Master: 9825670756d84f25af5639804ca661dc (127.8.14.67:35191)
I20260808 16:56:54.422753 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:56:54.425208 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.424096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48322 (local address 127.8.14.126:43045)
0808 16:56:54.424182 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:54.424184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:54.424260 (+ 76us) server_negotiation.cc:407] Connection header received
0808 16:56:54.424360 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:54.424361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:54.424365 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:54.424413 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:54.425053 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:54.425054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:54.425054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:54.425058 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:54.425070 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:54.425074 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:54.425117 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:54.425158 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 16:56:54.425169 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 16:56:54.426718 14966 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48322:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:56:54.428314 14966 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:56:54.432240 15678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.430861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34112 (local address 127.8.14.65:38085)
0808 16:56:54.431058 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:54.431062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:54.431072 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:56:54.431099 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:54.431101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:54.431111 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:54.431365 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:54.431910 (+ 545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:54.431914 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:54.431915 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:54.431919 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:54.431953 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:54.431968 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:54.432034 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:54.432135 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 16:56:54.432151 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":117,"threads_started":1}
I20260808 16:56:54.432266 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.430621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:38085 (local address 127.0.0.1:34112)
0808 16:56:54.430776 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:54.430785 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:54.430834 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:54.431220 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:54.431221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:54.431223 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:54.431695 (+ 472us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:56:54.431696 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:54.431711 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:56:54.431734 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:54.432029 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:54.432037 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:54.432039 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:56:54.432128 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 16:56:54.432143 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":76,"threads_started":1}
I20260808 16:56:54.432273 15671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.430411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:35191 (local address 127.0.0.1:54694)
0808 16:56:54.430555 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:54.430595 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:54.430652 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:54.431485 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:54.431491 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:54.431493 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:54.431857 (+ 364us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:56:54.431857 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:54.431865 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:56:54.431879 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:54.432061 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:54.432062 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:54.432063 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 16:56:54.432141 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 16:56:54.432156 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260808 16:56:54.432446 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.430563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54694 (local address 127.8.14.67:35191)
0808 16:56:54.431093 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:54.431097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:54.431107 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:56:54.431301 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:54.431307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:54.431314 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:54.431672 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:54.431934 (+ 262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:54.431937 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:54.431937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:54.431941 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:54.431968 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:54.431980 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:54.432035 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:54.432306 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 16:56:54.432322 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":70,"threads_started":1}
I20260808 16:56:54.432614 15672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.430518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45090 (local address 127.8.14.66:40937)
0808 16:56:54.430724 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:54.430731 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:54.430796 (+ 65us) server_negotiation.cc:407] Connection header received
0808 16:56:54.430842 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:54.430845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:54.430855 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:54.430969 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:54.431992 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:54.431998 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:54.431998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:54.432004 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:54.432034 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:54.432061 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:54.432150 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:54.432256 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 16:56:54.432271 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":84,"threads_started":1}
I20260808 16:56:54.432874 15670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.430425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:40937 (local address 127.0.0.1:45090)
0808 16:56:54.430702 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:54.430713 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:54.430779 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:54.430969 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:54.430975 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:54.430982 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:54.431866 (+ 884us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:56:54.431866 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:54.431873 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:56:54.431885 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:54.432176 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:54.432177 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:54.432178 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 16:56:54.432805 (+ 627us) client_negotiation.cc:240] Negotiation successful
0808 16:56:54.432819 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":80,"threads_started":1}
I20260808 16:56:54.433063 15152 tablet_service.cc:1467] Processing CreateTablet for tablet 5bfbfbaadefa4c29b243e175bdebd418 (DEFAULT_TABLE table=test-table [id=263f2520bd9348b0bfea0a61082c30d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:54.433111 15557 tablet_service.cc:1467] Processing CreateTablet for tablet 5bfbfbaadefa4c29b243e175bdebd418 (DEFAULT_TABLE table=test-table [id=263f2520bd9348b0bfea0a61082c30d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:54.433481 15152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bfbfbaadefa4c29b243e175bdebd418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:54.433483 15557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bfbfbaadefa4c29b243e175bdebd418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:54.436190 15679 tablet_bootstrap.cc:492] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8: Bootstrap starting.
I20260808 16:56:54.436215 15680 tablet_bootstrap.cc:492] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc: Bootstrap starting.
I20260808 16:56:54.436898 15680 tablet_bootstrap.cc:654] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:54.436946 15679 tablet_bootstrap.cc:654] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:54.437036 15358 tablet_service.cc:1467] Processing CreateTablet for tablet 5bfbfbaadefa4c29b243e175bdebd418 (DEFAULT_TABLE table=test-table [id=263f2520bd9348b0bfea0a61082c30d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:56:54.437230 15680 log.cc:824] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:54.437278 15679 log.cc:824] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:54.437391 15358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bfbfbaadefa4c29b243e175bdebd418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:54.437999 15680 tablet_bootstrap.cc:492] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc: No bootstrap required, opened a new log
I20260808 16:56:54.438006 15679 tablet_bootstrap.cc:492] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8: No bootstrap required, opened a new log
I20260808 16:56:54.438066 15679 ts_tablet_manager.cc:1397] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:56:54.438066 15680 ts_tablet_manager.cc:1397] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:56:54.439996 15683 tablet_bootstrap.cc:492] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2: Bootstrap starting.
I20260808 16:56:54.440522 15683 tablet_bootstrap.cc:654] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:54.440729 15683 log.cc:824] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:54.440742 15680 raft_consensus.cc:348] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9825670756d84f25af5639804ca661dc" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35191 } } peers { permanent_uuid: "38649115c1634e938110e0c4d901d8e2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40937 } } peers { permanent_uuid: "c74c34eb862a4712af375096109974d8" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38085 } }
I20260808 16:56:54.440742 15679 raft_consensus.cc:348] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9825670756d84f25af5639804ca661dc" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35191 } } peers { permanent_uuid: "38649115c1634e938110e0c4d901d8e2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40937 } } peers { permanent_uuid: "c74c34eb862a4712af375096109974d8" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38085 } }
I20260808 16:56:54.440897 15679 raft_consensus.cc:374] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:54.440897 15680 raft_consensus.cc:374] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:54.440934 15680 raft_consensus.cc:729] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9825670756d84f25af5639804ca661dc, State: Initialized, Role: FOLLOWER
I20260808 16:56:54.440934 15679 raft_consensus.cc:729] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c74c34eb862a4712af375096109974d8, State: Initialized, Role: FOLLOWER
I20260808 16:56:54.441054 15680 consensus_queue.cc:260] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9825670756d84f25af5639804ca661dc" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35191 } } peers { permanent_uuid: "38649115c1634e938110e0c4d901d8e2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40937 } } peers { permanent_uuid: "c74c34eb862a4712af375096109974d8" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38085 } }
I20260808 16:56:54.441318 15680 ts_tablet_manager.cc:1428] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 16:56:54.441358 15656 heartbeater.cc:503] Master 127.8.14.126:43045 was elected leader, sending a full tablet report...
I20260808 16:56:54.441468 15683 tablet_bootstrap.cc:492] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2: No bootstrap required, opened a new log
I20260808 16:56:54.441358 15679 consensus_queue.cc:260] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9825670756d84f25af5639804ca661dc" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35191 } } peers { permanent_uuid: "38649115c1634e938110e0c4d901d8e2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40937 } } peers { permanent_uuid: "c74c34eb862a4712af375096109974d8" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38085 } }
I20260808 16:56:54.441534 15683 ts_tablet_manager.cc:1397] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:56:54.441695 15679 ts_tablet_manager.cc:1428] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 16:56:54.441780 15262 heartbeater.cc:503] Master 127.8.14.126:43045 was elected leader, sending a full tablet report...
I20260808 16:56:54.443827 15683 raft_consensus.cc:348] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9825670756d84f25af5639804ca661dc" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35191 } } peers { permanent_uuid: "38649115c1634e938110e0c4d901d8e2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40937 } } peers { permanent_uuid: "c74c34eb862a4712af375096109974d8" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38085 } }
I20260808 16:56:54.443948 15683 raft_consensus.cc:374] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:54.443969 15683 raft_consensus.cc:729] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38649115c1634e938110e0c4d901d8e2, State: Initialized, Role: FOLLOWER
I20260808 16:56:54.444041 15683 consensus_queue.cc:260] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9825670756d84f25af5639804ca661dc" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35191 } } peers { permanent_uuid: "38649115c1634e938110e0c4d901d8e2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40937 } } peers { permanent_uuid: "c74c34eb862a4712af375096109974d8" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38085 } }
I20260808 16:56:54.444262 15683 ts_tablet_manager.cc:1428] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:56:54.444424 15449 heartbeater.cc:503] Master 127.8.14.126:43045 was elected leader, sending a full tablet report...
W20260808 16:56:54.552091 15451 tablet.cc:2369] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:56:54.659988 15692 raft_consensus.cc:482] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:56:54.660136 15692 raft_consensus.cc:504] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9825670756d84f25af5639804ca661dc" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35191 } } peers { permanent_uuid: "38649115c1634e938110e0c4d901d8e2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40937 } } peers { permanent_uuid: "c74c34eb862a4712af375096109974d8" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38085 } }
I20260808 16:56:54.660568 15692 leader_election.cc:288] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9825670756d84f25af5639804ca661dc (127.8.14.67:35191), c74c34eb862a4712af375096109974d8 (127.8.14.65:38085)
I20260808 16:56:54.661899 15452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.660748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:38085 (local address 127.8.14.66:42703)
0808 16:56:54.660811 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:54.660824 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:54.660866 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:54.661031 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:54.661032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:54.661034 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:54.661573 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:56:54.661574 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:54.661581 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:56:54.661593 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:54.661774 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:54.661775 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:54.661776 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 16:56:54.661819 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 16:56:54.661835 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 16:56:54.661907 15678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.660846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:42703 (local address 127.8.14.65:38085)
0808 16:56:54.660890 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:54.660892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:54.660902 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:56:54.660930 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:54.660930 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:54.660952 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:54.661014 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:54.661683 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:54.661685 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:54.661685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:54.661690 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:54.661711 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:54.661716 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:54.661777 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:54.661835 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 16:56:54.661848 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:56:54.662122 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.660851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:52999 (local address 127.8.14.67:35191)
0808 16:56:54.660900 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:54.660903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:54.661127 (+ 224us) server_negotiation.cc:407] Connection header received
0808 16:56:54.661157 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:54.661158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:54.661164 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:54.661232 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:54.661874 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:54.661875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:54.661876 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:54.661880 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:54.661895 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:54.661899 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:54.661947 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:54.662017 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 16:56:54.662035 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 16:56:54.662146 15693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.660753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:35191 (local address 127.8.14.66:52999)
0808 16:56:54.661009 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:54.661019 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:54.661085 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:54.661278 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:54.661279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:54.661281 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:54.661738 (+ 457us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:56:54.661739 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:54.661752 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:56:54.661763 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:54.661953 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:54.661954 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:54.661954 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 16:56:54.662004 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 16:56:54.662017 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":80,"thread_start_us":103,"threads_started":1}
I20260808 16:56:54.662307 15188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bfbfbaadefa4c29b243e175bdebd418" candidate_uuid: "38649115c1634e938110e0c4d901d8e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c74c34eb862a4712af375096109974d8" is_pre_election: true
I20260808 16:56:54.662474 15188 raft_consensus.cc:2454] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38649115c1634e938110e0c4d901d8e2 in term 0.
I20260808 16:56:54.662498 15586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bfbfbaadefa4c29b243e175bdebd418" candidate_uuid: "38649115c1634e938110e0c4d901d8e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9825670756d84f25af5639804ca661dc" is_pre_election: true
I20260808 16:56:54.662621 15586 raft_consensus.cc:2454] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38649115c1634e938110e0c4d901d8e2 in term 0.
I20260808 16:56:54.662731 15289 leader_election.cc:302] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38649115c1634e938110e0c4d901d8e2, c74c34eb862a4712af375096109974d8; no voters:
I20260808 16:56:54.662863 15692 raft_consensus.cc:2781] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:56:54.662945 15692 raft_consensus.cc:482] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:56:54.662969 15692 raft_consensus.cc:3037] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:54.663678 15692 raft_consensus.cc:504] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9825670756d84f25af5639804ca661dc" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35191 } } peers { permanent_uuid: "38649115c1634e938110e0c4d901d8e2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40937 } } peers { permanent_uuid: "c74c34eb862a4712af375096109974d8" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38085 } }
I20260808 16:56:54.663822 15692 leader_election.cc:288] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [CANDIDATE]: Term 1 election: Requested vote from peers 9825670756d84f25af5639804ca661dc (127.8.14.67:35191), c74c34eb862a4712af375096109974d8 (127.8.14.65:38085)
I20260808 16:56:54.663944 15586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bfbfbaadefa4c29b243e175bdebd418" candidate_uuid: "38649115c1634e938110e0c4d901d8e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9825670756d84f25af5639804ca661dc"
I20260808 16:56:54.663986 15188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bfbfbaadefa4c29b243e175bdebd418" candidate_uuid: "38649115c1634e938110e0c4d901d8e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c74c34eb862a4712af375096109974d8"
I20260808 16:56:54.664024 15586 raft_consensus.cc:3037] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:54.664065 15188 raft_consensus.cc:3037] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:54.664731 15586 raft_consensus.cc:2454] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38649115c1634e938110e0c4d901d8e2 in term 1.
I20260808 16:56:54.664742 15188 raft_consensus.cc:2454] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38649115c1634e938110e0c4d901d8e2 in term 1.
I20260808 16:56:54.664919 15293 leader_election.cc:302] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38649115c1634e938110e0c4d901d8e2, 9825670756d84f25af5639804ca661dc; no voters:
I20260808 16:56:54.665014 15692 raft_consensus.cc:2781] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:54.665175 15692 raft_consensus.cc:686] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [term 1 LEADER]: Becoming Leader. State: Replica: 38649115c1634e938110e0c4d901d8e2, State: Running, Role: LEADER
I20260808 16:56:54.665269 15692 consensus_queue.cc:237] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9825670756d84f25af5639804ca661dc" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35191 } } peers { permanent_uuid: "38649115c1634e938110e0c4d901d8e2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40937 } } peers { permanent_uuid: "c74c34eb862a4712af375096109974d8" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38085 } }
I20260808 16:56:54.666030 14967 catalog_manager.cc:5495] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38649115c1634e938110e0c4d901d8e2 (127.8.14.66). New cstate: current_term: 1 leader_uuid: "38649115c1634e938110e0c4d901d8e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9825670756d84f25af5639804ca661dc" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38649115c1634e938110e0c4d901d8e2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c74c34eb862a4712af375096109974d8" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38085 } health_report { overall_health: UNKNOWN } } }
W20260808 16:56:54.668345 15657 tablet.cc:2369] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:56:54.691255 15263 tablet.cc:2369] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:56:54.714426 15672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.713489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45098 (local address 127.8.14.66:40937)
0808 16:56:54.713546 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:54.713550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:54.713563 (+ 13us) server_negotiation.cc:407] Connection header received
0808 16:56:54.713596 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:54.713597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:54.713609 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:54.713660 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:54.714223 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:54.714224 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:54.714224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:54.714229 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:54.714245 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:54.714250 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:54.714302 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:54.714359 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 16:56:54.714372 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:56:54.716267 15188 raft_consensus.cc:1261] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8 [term 1 FOLLOWER]: Refusing update from remote peer 38649115c1634e938110e0c4d901d8e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:54.716286 15586 raft_consensus.cc:1261] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc [term 1 FOLLOWER]: Refusing update from remote peer 38649115c1634e938110e0c4d901d8e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:56:54.716609 15692 consensus_queue.cc:1035] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9825670756d84f25af5639804ca661dc" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:54.716759 15694 consensus_queue.cc:1035] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c74c34eb862a4712af375096109974d8" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:56:54.719022 15698 mvcc.cc:205] Tried to move back new op lower bound from 7316308847475113984 to 7316308847269441536. Current Snapshot: MvccSnapshot[applied={T|T < 7316308847475113984}]
I20260808 16:56:54.719924 15699 mvcc.cc:205] Tried to move back new op lower bound from 7316308847475113984 to 7316308847269441536. Current Snapshot: MvccSnapshot[applied={T|T < 7316308847475113984 or (T in {7316308847475113984})}]
I20260808 16:56:54.720036 15700 mvcc.cc:205] Tried to move back new op lower bound from 7316308847475113984 to 7316308847269441536. Current Snapshot: MvccSnapshot[applied={T|T < 7316308847475113984 or (T in {7316308847475113984})}]
I20260808 16:56:54.721351 14966 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:56:54.721437 14966 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:56:54.722184 14966 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18393d8b9f3f4ecd84483ddff4966ee7: Sending DeleteTablet for 3 replicas of tablet 5bfbfbaadefa4c29b243e175bdebd418
I20260808 16:56:54.722518 15358 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bfbfbaadefa4c29b243e175bdebd418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:54 UTC) from {username='slave'} at 127.0.0.1:45090
I20260808 16:56:54.722543 15152 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bfbfbaadefa4c29b243e175bdebd418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:54 UTC) from {username='slave'} at 127.0.0.1:34112
I20260808 16:56:54.722760 15713 tablet_replica.cc:330] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8: stopping tablet replica
I20260808 16:56:54.722875 15711 tablet_replica.cc:330] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2: stopping tablet replica
I20260808 16:56:54.722875 15557 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bfbfbaadefa4c29b243e175bdebd418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:56:54 UTC) from {username='slave'} at 127.0.0.1:54694
I20260808 16:56:54.723073 15711 raft_consensus.cc:2229] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:56:54.723143 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 15071
I20260808 16:56:54.723197 15711 raft_consensus.cc:2258] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:54.723178 15713 raft_consensus.cc:2229] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:56:54.723439 15713 raft_consensus.cc:2258] T 5bfbfbaadefa4c29b243e175bdebd418 P c74c34eb862a4712af375096109974d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:54.723649 15711 ts_tablet_manager.cc:1905] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:54.725865 15711 ts_tablet_manager.cc:1918] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:56:54.725957 15711 log.cc:1196] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/5bfbfbaadefa4c29b243e175bdebd418
I20260808 16:56:54.726260 15711 ts_tablet_manager.cc:1939] T 5bfbfbaadefa4c29b243e175bdebd418 P 38649115c1634e938110e0c4d901d8e2: Deleting consensus metadata
I20260808 16:56:54.726719 14940 catalog_manager.cc:4841] TS 38649115c1634e938110e0c4d901d8e2 (127.8.14.66:40937): tablet 5bfbfbaadefa4c29b243e175bdebd418 (table test-table [id=263f2520bd9348b0bfea0a61082c30d2]) successfully deleted
I20260808 16:56:54.731036 15714 tablet_replica.cc:330] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc: stopping tablet replica
I20260808 16:56:54.731149 15714 raft_consensus.cc:2229] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:56:54.731254 15714 raft_consensus.cc:2258] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:56:54.731736 15714 ts_tablet_manager.cc:1905] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:56:54.733362 15714 ts_tablet_manager.cc:1918] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:56:54.733438 15714 log.cc:1196] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/5bfbfbaadefa4c29b243e175bdebd418
I20260808 16:56:54.733717 15714 ts_tablet_manager.cc:1939] T 5bfbfbaadefa4c29b243e175bdebd418 P 9825670756d84f25af5639804ca661dc: Deleting consensus metadata
I20260808 16:56:54.734165 14945 catalog_manager.cc:4841] TS 9825670756d84f25af5639804ca661dc (127.8.14.67:35191): tablet 5bfbfbaadefa4c29b243e175bdebd418 (table test-table [id=263f2520bd9348b0bfea0a61082c30d2]) successfully deleted
W20260808 16:56:54.735196 14940 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.14.65:38085 (error 108)
I20260808 16:56:54.735419 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 15265
I20260808 16:56:54.737610 15670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.735576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:38085 (local address 127.0.0.1:34118)
0808 16:56:54.737527 (+ 1951us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:54.737574 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.65:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1916}
W20260808 16:56:54.737759 14940 catalog_manager.cc:4584] TS c74c34eb862a4712af375096109974d8 (127.8.14.65:38085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5bfbfbaadefa4c29b243e175bdebd418: Network error: Client connection negotiation failed: client connection to 127.8.14.65:38085: connect: Connection refused (error 111)
I20260808 16:56:54.751444 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 15453
I20260808 16:56:54.759979 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 14919
2026-08-08T16:56:54Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (875 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 16:56:54.784914 8249 mini_oidc.cc:152] Starting JWKS server
I20260808 16:56:54.786089 8249 webserver.cc:460] Webserver started at https://127.0.0.1:44487/ using document root <none> and password file <none>
I20260808 16:56:54.786214 8249 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 16:56:54.786408 8249 webserver.cc:460] Webserver started at http://127.0.0.1:42449/ using document root <none> and password file <none>
2026-08-08T16:56:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:54Z Disabled control of system clock
I20260808 16:56:54.795275 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:46353
--webserver_interface=127.8.14.126
--webserver_port=0
--builtin_ntp_servers=127.8.14.84:42367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:46353
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44487/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/testchainca.cert with env {}
W20260808 16:56:54.877219 15724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:54.877408 15724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:54.877429 15724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:54.878978 15724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:54.879017 15724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:54.879029 15724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:54.879040 15724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:54.879052 15724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 16:56:54.879063 15724 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 16:56:54.879073 15724 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44487/jwks.json
I20260808 16:56:54.880553 15724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:42367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:46353
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:46353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44487/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:54.880734 15724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:54.880883 15724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:54.885730 15732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:54.885720 15731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:54.885842 15724 server_base.cc:1034] running on GCE node
W20260808 16:56:54.885720 15738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:54.886179 15724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:54.886462 15724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:54.887604 15724 hybrid_clock.cc:648] HybridClock initialized: now 1786208214887594 us; error 32 us; skew 500 ppm
I20260808 16:56:54.888818 15724 webserver.cc:460] Webserver started at http://127.8.14.126:40053/ using document root <none> and password file <none>
I20260808 16:56:54.889055 15724 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:54.889101 15724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:54.889307 15724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:54.890331 15724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "09044c9efedf4b2f99d970ff6319c61c"
format_stamp: "Formatted at 2026-08-08 16:56:54 on dist-test-slave-1nl6"
server_key: "1ffb78d4c8beb6914ab015bc5ae62726"
server_key_iv: "d51e6be029325e746e40f11d33514273"
server_key_version: "encryptionkey@0"
I20260808 16:56:54.890712 15724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "09044c9efedf4b2f99d970ff6319c61c"
format_stamp: "Formatted at 2026-08-08 16:56:54 on dist-test-slave-1nl6"
server_key: "1ffb78d4c8beb6914ab015bc5ae62726"
server_key_iv: "d51e6be029325e746e40f11d33514273"
server_key_version: "encryptionkey@0"
I20260808 16:56:54.892062 15724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:56:54.892808 15748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:54.892979 15724 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:56:54.893070 15724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "09044c9efedf4b2f99d970ff6319c61c"
format_stamp: "Formatted at 2026-08-08 16:56:54 on dist-test-slave-1nl6"
server_key: "1ffb78d4c8beb6914ab015bc5ae62726"
server_key_iv: "d51e6be029325e746e40f11d33514273"
server_key_version: "encryptionkey@0"
I20260808 16:56:54.893124 15724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:54.917820 15724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:54.918051 15724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:54.918146 15724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:54.921765 15724 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:46353
I20260808 16:56:54.921840 15824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:46353 every 8 connection(s)
I20260808 16:56:54.922189 15724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:54.923241 15825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:54.925539 15825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c: Bootstrap starting.
I20260808 16:56:54.926143 15825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:54.926512 15825 log.cc:824] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:54.927275 15825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c: No bootstrap required, opened a new log
I20260808 16:56:54.928877 15825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09044c9efedf4b2f99d970ff6319c61c" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46353 } }
I20260808 16:56:54.929039 15825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:54.929090 15825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09044c9efedf4b2f99d970ff6319c61c, State: Initialized, Role: FOLLOWER
I20260808 16:56:54.929256 15825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09044c9efedf4b2f99d970ff6319c61c" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46353 } }
I20260808 16:56:54.929349 15825 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:54.929394 15825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:54.929446 15825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:54.929927 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 15724
I20260808 16:56:54.930045 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:56:54.930178 15825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09044c9efedf4b2f99d970ff6319c61c" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46353 } }
I20260808 16:56:54.930251 8249 external_mini_cluster.cc:1428] Setting key 35d152fee2949cbb609a3f9670cc0d0c
I20260808 16:56:54.930290 15825 leader_election.cc:302] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09044c9efedf4b2f99d970ff6319c61c; no voters:
I20260808 16:56:54.930506 15825 leader_election.cc:288] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:54.930537 15829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:54.930768 15825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:54.930881 15829 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [term 1 LEADER]: Becoming Leader. State: Replica: 09044c9efedf4b2f99d970ff6319c61c, State: Running, Role: LEADER
I20260808 16:56:54.931031 15829 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09044c9efedf4b2f99d970ff6319c61c" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46353 } }
I20260808 16:56:54.931470 15830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09044c9efedf4b2f99d970ff6319c61c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09044c9efedf4b2f99d970ff6319c61c" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46353 } } }
I20260808 16:56:54.931576 15830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:54.931990 15850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:54.931988 15836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09044c9efedf4b2f99d970ff6319c61c. Latest consensus state: current_term: 1 leader_uuid: "09044c9efedf4b2f99d970ff6319c61c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09044c9efedf4b2f99d970ff6319c61c" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46353 } } }
I20260808 16:56:54.932152 15836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:54.932757 15850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:54.934260 15850 catalog_manager.cc:1347] Generated new cluster ID: adde55aed68b47699a5330d7ba59b8e0
I20260808 16:56:54.934325 15850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:54.936436 15835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.930712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59014 (local address 127.8.14.126:46353)
0808 16:56:54.930893 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:54.930924 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:54.930957 (+ 33us) server_negotiation.cc:407] Connection header received
0808 16:56:54.931151 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:54.931160 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:54.931224 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:54.931637 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:54.931755 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:54.933203 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:54.934317 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:54.934922 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:54.935035 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:54.935757 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:54.935761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:54.935761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:54.935964 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:54.936003 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:54.936017 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:54.936077 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:54.936174 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 16:56:54.936244 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
I20260808 16:56:54.939440 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:54.953814 15850 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:54.954299 15850 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:54.957079 15850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09044c9efedf4b2f99d970ff6319c61c: Generated new TSK 0
I20260808 16:56:54.957245 15850 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 16:56:54.990599 15873 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 16:56:54.990724 15873 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 16:56:54.990794 15835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:54.967822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59020 (local address 127.8.14.126:46353)
0808 16:56:54.967882 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:54.967884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:54.968021 (+ 137us) server_negotiation.cc:407] Connection header received
0808 16:56:54.968118 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:54.968118 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:54.968128 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:54.968184 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0808 16:56:54.968356 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:54.968808 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:54.969898 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:54.970085 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:54.970133 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:54.970377 (+ 244us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 16:56:54.990534 (+ 20157us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 16:56:54.990709 (+ 175us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59020: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 16:56:54.990851 15835 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59020: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 16:56:54.991187 15872 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:54.967742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46353 (local address 127.0.0.1:59020)
0808 16:56:54.967948 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:54.967960 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:54.968032 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:54.968221 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:54.968222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:54.968224 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 16:56:54.968294 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:54.968296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:54.968885 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:54.968886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:54.969813 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:54.969817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:54.970164 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:54.970165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:54.970246 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:54.970271 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 16:56:54.990667 (+ 20396us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 16:56:54.990795 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.14.126:46353: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":106,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.14.126:46353: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 16:56:54.991698 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 15724
2026-08-08T16:56:55Z chronyd exiting
I20260808 16:56:55.042160 8249 test_util.cc:175] -----------------------------------------------
I20260808 16:56:55.042239 8249 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786208190438830-8249-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (258 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 16:56:55.043097 8249 mini_oidc.cc:152] Starting JWKS server
I20260808 16:56:55.043782 8249 webserver.cc:460] Webserver started at https://127.0.0.1:35093/ using document root <none> and password file <none>
I20260808 16:56:55.043831 8249 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 16:56:55.044001 8249 webserver.cc:460] Webserver started at http://127.0.0.1:46573/ using document root <none> and password file <none>
2026-08-08T16:56:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:55Z Disabled control of system clock
I20260808 16:56:55.052948 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:42337
--webserver_interface=127.8.14.126
--webserver_port=0
--builtin_ntp_servers=127.8.14.84:46287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:42337
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35093/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/testchainca.cert with env {}
W20260808 16:56:55.135895 15880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:55.136085 15880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:55.136106 15880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:55.137205 15880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:55.137244 15880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:55.137256 15880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:55.137269 15880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:55.137280 15880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 16:56:55.137291 15880 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 16:56:55.137307 15880 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35093/jwks.json
I20260808 16:56:55.138820 15880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:46287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:42337
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:42337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35093/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:55.139079 15880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:55.139249 15880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:55.143893 15887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:55.143936 15889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:55.143944 15891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:55.144109 15880 server_base.cc:1034] running on GCE node
I20260808 16:56:55.144418 15880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:55.144731 15880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:55.145877 15880 hybrid_clock.cc:648] HybridClock initialized: now 1786208215145857 us; error 38 us; skew 500 ppm
I20260808 16:56:55.147182 15880 webserver.cc:460] Webserver started at http://127.8.14.126:33291/ using document root <none> and password file <none>
I20260808 16:56:55.147398 15880 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:55.147440 15880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:55.147526 15880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:55.148473 15880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "51ad9f69bc9c44feafb7a0ee571f122f"
format_stamp: "Formatted at 2026-08-08 16:56:55 on dist-test-slave-1nl6"
server_key: "e5073533917f0c7bf714b320a778658b"
server_key_iv: "e3ccbecb55ba79877e65d97759fa4c66"
server_key_version: "encryptionkey@0"
I20260808 16:56:55.148748 15880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "51ad9f69bc9c44feafb7a0ee571f122f"
format_stamp: "Formatted at 2026-08-08 16:56:55 on dist-test-slave-1nl6"
server_key: "e5073533917f0c7bf714b320a778658b"
server_key_iv: "e3ccbecb55ba79877e65d97759fa4c66"
server_key_version: "encryptionkey@0"
I20260808 16:56:55.150020 15880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:55.150655 15902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:55.150837 15880 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:55.150954 15880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "51ad9f69bc9c44feafb7a0ee571f122f"
format_stamp: "Formatted at 2026-08-08 16:56:55 on dist-test-slave-1nl6"
server_key: "e5073533917f0c7bf714b320a778658b"
server_key_iv: "e3ccbecb55ba79877e65d97759fa4c66"
server_key_version: "encryptionkey@0"
I20260808 16:56:55.151042 15880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:55.174320 15880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:55.174594 15880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:55.174743 15880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:55.178467 15880 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:42337
I20260808 16:56:55.178536 15982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:42337 every 8 connection(s)
I20260808 16:56:55.178848 15880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:55.179785 15983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:55.181921 15983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f: Bootstrap starting.
I20260808 16:56:55.182510 15983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:55.182806 15983 log.cc:824] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:55.183679 15983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f: No bootstrap required, opened a new log
I20260808 16:56:55.185232 15983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ad9f69bc9c44feafb7a0ee571f122f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42337 } }
I20260808 16:56:55.185389 15983 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:55.185441 15983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51ad9f69bc9c44feafb7a0ee571f122f, State: Initialized, Role: FOLLOWER
I20260808 16:56:55.185631 15983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ad9f69bc9c44feafb7a0ee571f122f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42337 } }
I20260808 16:56:55.185736 15983 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:55.185782 15983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:55.185844 15983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:55.186354 15983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ad9f69bc9c44feafb7a0ee571f122f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42337 } }
I20260808 16:56:55.186476 15983 leader_election.cc:302] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51ad9f69bc9c44feafb7a0ee571f122f; no voters:
I20260808 16:56:55.186661 15983 leader_election.cc:288] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:55.186692 15986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:55.186892 15986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [term 1 LEADER]: Becoming Leader. State: Replica: 51ad9f69bc9c44feafb7a0ee571f122f, State: Running, Role: LEADER
I20260808 16:56:55.186954 15983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:55.187007 15986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ad9f69bc9c44feafb7a0ee571f122f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42337 } }
I20260808 16:56:55.187433 15987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51ad9f69bc9c44feafb7a0ee571f122f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ad9f69bc9c44feafb7a0ee571f122f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42337 } } }
I20260808 16:56:55.187579 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 15880
I20260808 16:56:55.187556 15988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51ad9f69bc9c44feafb7a0ee571f122f. Latest consensus state: current_term: 1 leader_uuid: "51ad9f69bc9c44feafb7a0ee571f122f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ad9f69bc9c44feafb7a0ee571f122f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42337 } } }
I20260808 16:56:55.187636 15988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:55.187687 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:56:55.187589 15987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:55.187896 8249 external_mini_cluster.cc:1428] Setting key cf2d1f19bb552651dd3e990a8d524fa1
W20260808 16:56:55.188885 16016 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:56:55.188953 16016 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:56:55.189003 16004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:55.189182 16004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:55.190672 16004 catalog_manager.cc:1347] Generated new cluster ID: e8a43a457a3244998589292ed7cb6bd0
I20260808 16:56:55.190781 16004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:55.193717 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:55.188344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41204 (local address 127.8.14.126:42337)
0808 16:56:55.188668 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:55.188676 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:55.188707 (+ 31us) server_negotiation.cc:407] Connection header received
0808 16:56:55.188838 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:55.188846 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:55.188914 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:55.189044 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:55.189360 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.190458 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.191602 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.192240 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.192370 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:55.193026 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:55.193030 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:55.193031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:55.193235 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:55.193275 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:55.193290 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:55.193355 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:55.193428 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 16:56:55.193508 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":91,"threads_started":1}
I20260808 16:56:55.194653 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:55.204227 16004 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:55.204690 16004 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:55.212843 16004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51ad9f69bc9c44feafb7a0ee571f122f: Generated new TSK 0
I20260808 16:56:55.213006 16004 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 16:56:55.236430 16033 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 16:56:55.236500 16033 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 16:56:55.236792 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:55.222752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41210 (local address 127.8.14.126:42337)
0808 16:56:55.222802 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:55.222804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:55.222922 (+ 118us) server_negotiation.cc:407] Connection header received
0808 16:56:55.223000 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:55.223000 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:55.223013 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:55.223092 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0808 16:56:55.223209 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.223986 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.224717 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.225053 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.225125 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:55.225281 (+ 156us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 16:56:55.236590 (+ 11309us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35093/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 16:56:55.236719 (+ 129us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41210: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35093/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 16:56:55.236848 16017 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41210: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35093/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 16:56:55.236856 16030 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:55.222680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42337 (local address 127.0.0.1:41210)
0808 16:56:55.222822 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:55.222832 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:55.222931 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:55.223098 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:55.223099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:55.223101 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 16:56:55.223144 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:55.223146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.224069 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.224070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:55.224640 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:55.224642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.225145 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.225146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:55.225185 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:55.225190 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 16:56:55.236682 (+ 11492us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35093/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 16:56:55.236761 (+ 79us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.14.126:42337: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35093/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":91,"threads_started":1}
I20260808 16:56:55.237085 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 15880
2026-08-08T16:56:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (239 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 16:56:55.282464 8249 mini_oidc.cc:152] Starting JWKS server
I20260808 16:56:55.283219 8249 webserver.cc:460] Webserver started at https://127.0.0.1:43427/ using document root <none> and password file <none>
I20260808 16:56:55.283280 8249 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 16:56:55.283455 8249 webserver.cc:460] Webserver started at http://127.0.0.1:34369/ using document root <none> and password file <none>
2026-08-08T16:56:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:55Z Disabled control of system clock
I20260808 16:56:55.292258 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:32945
--webserver_interface=127.8.14.126
--webserver_port=0
--builtin_ntp_servers=127.8.14.84:38739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:32945
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43427/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 16:56:55.373700 16045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:55.373895 16045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:55.373917 16045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:55.375137 16045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:55.375180 16045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:55.375192 16045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:55.375233 16045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:55.375262 16045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 16:56:55.375293 16045 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 16:56:55.375319 16045 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43427/jwks.json
I20260808 16:56:55.376763 16045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:38739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:32945
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:32945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43427/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:55.377012 16045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:55.377295 16045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:55.382052 16052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:55.382067 16051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:55.382071 16055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:55.382087 16045 server_base.cc:1034] running on GCE node
I20260808 16:56:55.382529 16045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:55.382790 16045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:55.383942 16045 hybrid_clock.cc:648] HybridClock initialized: now 1786208215383907 us; error 53 us; skew 500 ppm
I20260808 16:56:55.385106 16045 webserver.cc:460] Webserver started at http://127.8.14.126:43385/ using document root <none> and password file <none>
I20260808 16:56:55.385372 16045 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:55.385417 16045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:55.385583 16045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:55.386626 16045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "265c57d1215d45949c5f92a217f04e5e"
format_stamp: "Formatted at 2026-08-08 16:56:55 on dist-test-slave-1nl6"
server_key: "2af745d7c88d7436236e17f78cdcfb0a"
server_key_iv: "b4c2e41c0d4e847ff04eca9738255509"
server_key_version: "encryptionkey@0"
I20260808 16:56:55.387032 16045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "265c57d1215d45949c5f92a217f04e5e"
format_stamp: "Formatted at 2026-08-08 16:56:55 on dist-test-slave-1nl6"
server_key: "2af745d7c88d7436236e17f78cdcfb0a"
server_key_iv: "b4c2e41c0d4e847ff04eca9738255509"
server_key_version: "encryptionkey@0"
I20260808 16:56:55.388298 16045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:55.389017 16063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:55.389168 16045 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:55.389256 16045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "265c57d1215d45949c5f92a217f04e5e"
format_stamp: "Formatted at 2026-08-08 16:56:55 on dist-test-slave-1nl6"
server_key: "2af745d7c88d7436236e17f78cdcfb0a"
server_key_iv: "b4c2e41c0d4e847ff04eca9738255509"
server_key_version: "encryptionkey@0"
I20260808 16:56:55.389317 16045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:55.404438 16045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:55.404657 16045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:55.404753 16045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:55.408414 16045 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:32945
I20260808 16:56:55.408491 16142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:32945 every 8 connection(s)
I20260808 16:56:55.408768 16045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:55.409454 16143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:55.412211 16143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e: Bootstrap starting.
I20260808 16:56:55.412781 16143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:55.413081 16143 log.cc:824] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:55.413739 16143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e: No bootstrap required, opened a new log
I20260808 16:56:55.415370 16143 raft_consensus.cc:348] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265c57d1215d45949c5f92a217f04e5e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 32945 } }
I20260808 16:56:55.415520 16143 raft_consensus.cc:374] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:55.415566 16143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 265c57d1215d45949c5f92a217f04e5e, State: Initialized, Role: FOLLOWER
I20260808 16:56:55.415748 16143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265c57d1215d45949c5f92a217f04e5e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 32945 } }
I20260808 16:56:55.415853 16143 raft_consensus.cc:388] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:55.415899 16143 raft_consensus.cc:482] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:55.415953 16143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:55.416496 16143 raft_consensus.cc:504] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265c57d1215d45949c5f92a217f04e5e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 32945 } }
I20260808 16:56:55.416567 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 16045
I20260808 16:56:55.416612 16143 leader_election.cc:302] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 265c57d1215d45949c5f92a217f04e5e; no voters:
I20260808 16:56:55.416733 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:56:55.416844 16143 leader_election.cc:288] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:55.416880 16149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:55.416934 8249 external_mini_cluster.cc:1428] Setting key 00dd6ffde2a75e1c09443ddda6f6d120
I20260808 16:56:55.416999 16149 raft_consensus.cc:686] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [term 1 LEADER]: Becoming Leader. State: Replica: 265c57d1215d45949c5f92a217f04e5e, State: Running, Role: LEADER
I20260808 16:56:55.417107 16143 sys_catalog.cc:564] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:55.417094 16149 consensus_queue.cc:237] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265c57d1215d45949c5f92a217f04e5e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 32945 } }
I20260808 16:56:55.417608 16151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "265c57d1215d45949c5f92a217f04e5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265c57d1215d45949c5f92a217f04e5e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 32945 } } }
I20260808 16:56:55.417748 16151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:55.417587 16149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 265c57d1215d45949c5f92a217f04e5e. Latest consensus state: current_term: 1 leader_uuid: "265c57d1215d45949c5f92a217f04e5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "265c57d1215d45949c5f92a217f04e5e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 32945 } } }
I20260808 16:56:55.417881 16149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:55.418690 16166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:55.419175 16166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:55.421078 16166 catalog_manager.cc:1347] Generated new cluster ID: ccb849e4d3ea44eb86114a9b9e33f755
I20260808 16:56:55.421132 16166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:55.423178 16156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:55.417452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33554 (local address 127.8.14.126:32945)
0808 16:56:55.417737 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:55.417748 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:55.417787 (+ 39us) server_negotiation.cc:407] Connection header received
0808 16:56:55.417918 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:55.417925 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:55.418000 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:55.418133 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:55.418460 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.419944 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.421027 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.421606 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.421731 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:55.422433 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:55.422437 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:55.422437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:55.422643 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:55.422681 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:55.422693 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:55.422753 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:55.422854 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 16:56:55.422959 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":66,"threads_started":1}
I20260808 16:56:55.424052 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:55.436269 16166 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:55.436975 16166 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:55.447489 16166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 265c57d1215d45949c5f92a217f04e5e: Generated new TSK 0
I20260808 16:56:55.447661 16166 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:55.456403 16156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:55.453039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33570 (local address 127.8.14.126:32945)
0808 16:56:55.453106 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:55.453108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:55.453233 (+ 125us) server_negotiation.cc:407] Connection header received
0808 16:56:55.453323 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:55.453332 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:55.453344 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:55.453428 (+ 84us) server_negotiation.cc:226] Negotiated authn=JWT
0808 16:56:55.453568 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.454042 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.454835 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.455082 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.455155 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:55.455360 (+ 205us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 16:56:55.456221 (+ 861us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43427/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 16:56:55.456345 (+ 124us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33570: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43427/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 16:56:55.456454 16156 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33570: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43427/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 16:56:55.456727 16185 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:55.452965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:32945 (local address 127.0.0.1:33570)
0808 16:56:55.453161 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:55.453175 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:55.453247 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:55.453429 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:55.453430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:55.453432 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 16:56:55.453490 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:55.453492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.454130 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.454132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:55.454732 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:55.454733 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.455164 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.455165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:55.455222 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:55.455227 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 16:56:55.456331 (+ 1104us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43427/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 16:56:55.456429 (+ 98us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.14.126:32945: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43427/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":108,"threads_started":1}
I20260808 16:56:55.456892 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 16045
E20260808 16:56:55.470479 16186 webserver.cc:533] Webserver: sslize failed:
E20260808 16:56:55.470561 16186 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T16:56:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (216 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T16:56:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:55Z Disabled control of system clock
I20260808 16:56:55.507176 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:41371
--webserver_interface=127.8.14.126
--webserver_port=0
--builtin_ntp_servers=127.8.14.84:41315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:41371
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/testchainca.cert with env {}
W20260808 16:56:55.588557 16194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:55.588769 16194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:55.588806 16194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:55.589968 16194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:55.590029 16194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:55.590051 16194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:55.590082 16194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:55.590111 16194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 16:56:55.590137 16194 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 16:56:55.590163 16194 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 16:56:55.591694 16194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:41371
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:41371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:55.591957 16194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:55.592183 16194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:55.596866 16201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:55.596875 16202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:55.596932 16194 server_base.cc:1034] running on GCE node
W20260808 16:56:55.596866 16205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:55.597319 16194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:55.597625 16194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:55.598793 16194 hybrid_clock.cc:648] HybridClock initialized: now 1786208215598769 us; error 35 us; skew 500 ppm
I20260808 16:56:55.600030 16194 webserver.cc:460] Webserver started at http://127.8.14.126:34239/ using document root <none> and password file <none>
I20260808 16:56:55.600296 16194 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:55.600379 16194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:55.600528 16194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:55.601553 16194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "e420cf1a92a14868af6477490ff44cf1"
format_stamp: "Formatted at 2026-08-08 16:56:55 on dist-test-slave-1nl6"
server_key: "7ce06cac20c679fee3425a85e2c40d20"
server_key_iv: "41ab7613ebdab09fecc5144c43250b53"
server_key_version: "encryptionkey@0"
I20260808 16:56:55.601908 16194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "e420cf1a92a14868af6477490ff44cf1"
format_stamp: "Formatted at 2026-08-08 16:56:55 on dist-test-slave-1nl6"
server_key: "7ce06cac20c679fee3425a85e2c40d20"
server_key_iv: "41ab7613ebdab09fecc5144c43250b53"
server_key_version: "encryptionkey@0"
I20260808 16:56:55.603246 16194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:56:55.604005 16216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:55.604176 16194 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:55.604290 16194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "e420cf1a92a14868af6477490ff44cf1"
format_stamp: "Formatted at 2026-08-08 16:56:55 on dist-test-slave-1nl6"
server_key: "7ce06cac20c679fee3425a85e2c40d20"
server_key_iv: "41ab7613ebdab09fecc5144c43250b53"
server_key_version: "encryptionkey@0"
I20260808 16:56:55.604372 16194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:55.650730 16194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:55.651114 16194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:55.651280 16194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:55.655150 16194 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:41371
I20260808 16:56:55.655192 16296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:41371 every 8 connection(s)
I20260808 16:56:55.655582 16194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:55.656558 16298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:55.658780 16298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1: Bootstrap starting.
I20260808 16:56:55.659435 16298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:55.659720 16298 log.cc:824] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:55.660434 16298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1: No bootstrap required, opened a new log
I20260808 16:56:55.661657 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 16194
I20260808 16:56:55.661777 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:56:55.661975 8249 external_mini_cluster.cc:1428] Setting key 56ca46860aec53d4c96870afc8ee270a
I20260808 16:56:55.662250 16298 raft_consensus.cc:348] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e420cf1a92a14868af6477490ff44cf1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41371 } }
I20260808 16:56:55.662441 16298 raft_consensus.cc:374] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:55.662488 16298 raft_consensus.cc:729] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e420cf1a92a14868af6477490ff44cf1, State: Initialized, Role: FOLLOWER
I20260808 16:56:55.662611 16298 consensus_queue.cc:260] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e420cf1a92a14868af6477490ff44cf1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41371 } }
I20260808 16:56:55.662705 16298 raft_consensus.cc:388] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:55.662752 16298 raft_consensus.cc:482] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:55.662801 16298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:55.663715 16298 raft_consensus.cc:504] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e420cf1a92a14868af6477490ff44cf1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41371 } }
I20260808 16:56:55.663847 16298 leader_election.cc:302] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e420cf1a92a14868af6477490ff44cf1; no voters:
I20260808 16:56:55.664013 16298 leader_election.cc:288] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:55.664062 16304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:55.664189 16304 raft_consensus.cc:686] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [term 1 LEADER]: Becoming Leader. State: Replica: e420cf1a92a14868af6477490ff44cf1, State: Running, Role: LEADER
I20260808 16:56:55.664310 16298 sys_catalog.cc:564] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:55.664290 16304 consensus_queue.cc:237] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e420cf1a92a14868af6477490ff44cf1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41371 } }
I20260808 16:56:55.664659 16304 sys_catalog.cc:455] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e420cf1a92a14868af6477490ff44cf1. Latest consensus state: current_term: 1 leader_uuid: "e420cf1a92a14868af6477490ff44cf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e420cf1a92a14868af6477490ff44cf1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41371 } } }
I20260808 16:56:55.664752 16304 sys_catalog.cc:458] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:55.664979 16313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:55.665689 16313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:55.665841 16309 sys_catalog.cc:455] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e420cf1a92a14868af6477490ff44cf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e420cf1a92a14868af6477490ff44cf1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41371 } } }
I20260808 16:56:55.665972 16309 sys_catalog.cc:458] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1 [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:55.667615 16313 catalog_manager.cc:1347] Generated new cluster ID: 6a4e69396ce849e1a83b5ece6adf6cfa
I20260808 16:56:55.667672 16313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:55.667868 16303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:55.662487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46924 (local address 127.8.14.126:41371)
0808 16:56:55.662688 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:55.662695 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:55.662720 (+ 25us) server_negotiation.cc:407] Connection header received
0808 16:56:55.662870 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:55.662876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:55.662964 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:55.663080 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:55.663341 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.664421 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.665545 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.666166 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.666289 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:55.667074 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:55.667077 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:55.667078 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:56:55.667316 (+ 238us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:55.667356 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:56:55.667371 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:55.667437 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:55.667573 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 16:56:55.667656 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260808 16:56:55.668850 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:56:55.675334 16313 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:55.675830 16313 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:55.680804 16313 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e420cf1a92a14868af6477490ff44cf1: Generated new TSK 0
I20260808 16:56:55.680969 16313 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:56:56.160817 16303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:55.688336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46926 (local address 127.8.14.126:41371)
0808 16:56:55.688374 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:55.688376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:55.688496 (+ 120us) server_negotiation.cc:407] Connection header received
0808 16:56:55.688573 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:55.688574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:55.688586 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:55.688664 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0808 16:56:55.688807 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.689253 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.690454 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.690629 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.690688 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:55.690878 (+ 190us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 16:56:56.160582 (+469704us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 16:56:56.160755 (+ 173us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46926: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 16:56:56.160884 16303 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46926: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 16:56:56.161024 16338 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:55.688263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:41371 (local address 127.0.0.1:46926)
0808 16:56:55.688418 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:55.688428 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:55.688514 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:55.688672 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:55.688673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:55.688674 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0808 16:56:55.688729 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:55.688731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.689386 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.689388 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:55.690344 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:55.690347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:55.690712 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:55.690713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:55.690774 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:55.690779 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 16:56:56.160733 (+469954us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 16:56:56.160862 (+ 129us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.14.126:41371: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
I20260808 16:56:56.161404 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 16194
2026-08-08T16:56:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (686 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (90 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (90 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:56 dist-test-slave-1nl6 krb5kdc[16359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:56 dist-test-slave-1nl6 krb5kdc[16359](info): set up 2 sockets
Aug 08 16:56:56 dist-test-slave-1nl6 krb5kdc[16359](info): commencing operation
krb5kdc: starting...
W20260808 16:56:57.556192 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:56:57 dist-test-slave-1nl6 krb5kdc[16359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:56:57.671586 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:46859
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:43583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:46859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:57.752452 16377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:57.752641 16377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:57.752664 16377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:57.753750 16377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:57.753783 16377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:57.753798 16377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:57.753809 16377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:57.753821 16377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:57.755198 16377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:43583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:46859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:46859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:57.755384 16377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:57.755548 16377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:57.760232 16384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:57.760221 16391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:57.760226 16386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:57.760241 16377 server_base.cc:1034] running on GCE node
I20260808 16:56:57.760740 16377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:57.761032 16377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:57.762168 16377 hybrid_clock.cc:648] HybridClock initialized: now 1786208217762150 us; error 41 us; skew 500 ppm
Aug 08 16:56:57 dist-test-slave-1nl6 krb5kdc[16359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208217, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:57.764628 16377 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:56:57.765067 16377 webserver.cc:460] Webserver started at http://127.8.14.126:38523/ using document root <none> and password file <none>
I20260808 16:56:57.765264 16377 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:57.765307 16377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:57.765394 16377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:57.766326 16377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "7dd3d3bd996449b9958c77d062e7f80d"
format_stamp: "Formatted at 2026-08-08 16:56:57 on dist-test-slave-1nl6"
server_key: "77e2416b6a8ffcb9f1f005026d7b6d51"
server_key_iv: "6c71a448c842e5f0a57734e773a80d71"
server_key_version: "encryptionkey@0"
I20260808 16:56:57.766613 16377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "7dd3d3bd996449b9958c77d062e7f80d"
format_stamp: "Formatted at 2026-08-08 16:56:57 on dist-test-slave-1nl6"
server_key: "77e2416b6a8ffcb9f1f005026d7b6d51"
server_key_iv: "6c71a448c842e5f0a57734e773a80d71"
server_key_version: "encryptionkey@0"
I20260808 16:56:57.767935 16377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:56:57.768553 16403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:57.768729 16377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:56:57.768803 16377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "7dd3d3bd996449b9958c77d062e7f80d"
format_stamp: "Formatted at 2026-08-08 16:56:57 on dist-test-slave-1nl6"
server_key: "77e2416b6a8ffcb9f1f005026d7b6d51"
server_key_iv: "6c71a448c842e5f0a57734e773a80d71"
server_key_version: "encryptionkey@0"
I20260808 16:56:57.768855 16377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:57.778893 16377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:57.780582 16377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:57.780684 16377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:57.784543 16377 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:46859
I20260808 16:56:57.784607 16485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:46859 every 8 connection(s)
I20260808 16:56:57.784922 16377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:57.785749 16486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:57.786046 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 16377
I20260808 16:56:57.786132 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:56:57.786336 8249 external_mini_cluster.cc:1428] Setting key 5dc86b4140a5d693dbda2f284751477b
I20260808 16:56:57.789427 16486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d: Bootstrap starting.
I20260808 16:56:57.790091 16486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:57.790385 16486 log.cc:824] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:57.791175 16486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d: No bootstrap required, opened a new log
Aug 08 16:56:57 dist-test-slave-1nl6 krb5kdc[16359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:57.793768 16486 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd3d3bd996449b9958c77d062e7f80d" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46859 } }
I20260808 16:56:57.793921 16486 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:57.793987 16486 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd3d3bd996449b9958c77d062e7f80d, State: Initialized, Role: FOLLOWER
I20260808 16:56:57.794154 16486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd3d3bd996449b9958c77d062e7f80d" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46859 } }
I20260808 16:56:57.794256 16486 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:57.794315 16486 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:57.794368 16486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:57.794890 16486 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd3d3bd996449b9958c77d062e7f80d" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46859 } }
I20260808 16:56:57.795045 16486 leader_election.cc:302] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7dd3d3bd996449b9958c77d062e7f80d; no voters:
I20260808 16:56:57.795213 16486 leader_election.cc:288] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:57.795284 16493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:57.795464 16486 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:57.795490 16493 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [term 1 LEADER]: Becoming Leader. State: Replica: 7dd3d3bd996449b9958c77d062e7f80d, State: Running, Role: LEADER
I20260808 16:56:57.795657 16493 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd3d3bd996449b9958c77d062e7f80d" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46859 } }
I20260808 16:56:57.796087 16493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dd3d3bd996449b9958c77d062e7f80d. Latest consensus state: current_term: 1 leader_uuid: "7dd3d3bd996449b9958c77d062e7f80d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd3d3bd996449b9958c77d062e7f80d" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46859 } } }
I20260808 16:56:57.796176 16494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7dd3d3bd996449b9958c77d062e7f80d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd3d3bd996449b9958c77d062e7f80d" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46859 } } }
I20260808 16:56:57.796568 16493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:57.796591 16494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d [sys.catalog]: This master's current role is: LEADER
W20260808 16:56:57.797343 16514 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:56:57.797402 16514 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:56:57.797456 16515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:57.797636 16515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:57.798267 16490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:57.786856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34838 (local address 127.8.14.126:46859)
0808 16:56:57.787048 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:57.787059 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:57.787093 (+ 34us) server_negotiation.cc:407] Connection header received
0808 16:56:57.787569 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:57.787578 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:57.787648 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:57.787799 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:57.788550 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:57.789904 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:57.790759 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:57.791284 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:57.791397 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:57.794005 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:57.794009 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:57.794009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:57.794019 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:57.796176 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:57.796542 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:57.796545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:57.796546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:57.796670 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:57.796909 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:57.796910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:57.796911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:57.797259 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:57.797398 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:57.797927 (+ 529us) server_negotiation.cc:299] Negotiation successful
0808 16:56:57.798066 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":107,"threads_started":1}
I20260808 16:56:57.799010 16515 catalog_manager.cc:1347] Generated new cluster ID: 41d51dc9a39946d09c68866fe1da6b83
I20260808 16:56:57.799083 16515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:56:57.809698 16515 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:57.810191 16515 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:57.815287 16515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7dd3d3bd996449b9958c77d062e7f80d: Generated new TSK 0
I20260808 16:56:57.815456 16515 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 16:56:57.841655 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46859
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:43583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:57.923209 16524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:57.923398 16524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:57.923418 16524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:57.924474 16524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:57.924528 16524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:57.924553 16524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:56:57.926024 16524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:43583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:57.926208 16524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:57.926399 16524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:57.931275 16531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:57.931275 16534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:57.931329 16532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:57.931463 16524 server_base.cc:1034] running on GCE node
I20260808 16:56:57.931792 16524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:57.932029 16524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:57.933197 16524 hybrid_clock.cc:648] HybridClock initialized: now 1786208217933181 us; error 47 us; skew 500 ppm
Aug 08 16:56:57 dist-test-slave-1nl6 krb5kdc[16359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208217, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:57.935787 16524 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:56:57.936290 16524 webserver.cc:460] Webserver started at http://127.8.14.65:32977/ using document root <none> and password file <none>
I20260808 16:56:57.936512 16524 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:57.936578 16524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:57.936754 16524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:57.937774 16524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "4bf9055714bd4252be25ae2ab06064da"
format_stamp: "Formatted at 2026-08-08 16:56:57 on dist-test-slave-1nl6"
server_key: "bc4c0ee222a5d0b82fc4d09c47969e8a"
server_key_iv: "ebf766e97b5cc5d5dc16c0d314a63b52"
server_key_version: "encryptionkey@0"
I20260808 16:56:57.938149 16524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "4bf9055714bd4252be25ae2ab06064da"
format_stamp: "Formatted at 2026-08-08 16:56:57 on dist-test-slave-1nl6"
server_key: "bc4c0ee222a5d0b82fc4d09c47969e8a"
server_key_iv: "ebf766e97b5cc5d5dc16c0d314a63b52"
server_key_version: "encryptionkey@0"
I20260808 16:56:57.939538 16524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:57.940272 16541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:57.940510 16524 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:56:57.940635 16524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "4bf9055714bd4252be25ae2ab06064da"
format_stamp: "Formatted at 2026-08-08 16:56:57 on dist-test-slave-1nl6"
server_key: "bc4c0ee222a5d0b82fc4d09c47969e8a"
server_key_iv: "ebf766e97b5cc5d5dc16c0d314a63b52"
server_key_version: "encryptionkey@0"
I20260808 16:56:57.940722 16524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:57.957615 16524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:57.959550 16524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:57.959733 16524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:57.959997 16524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:57.960318 16524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:57.960405 16524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:57.960448 16524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:57.960477 16524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:57.966732 16524 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:35045
I20260808 16:56:57.966766 16709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:35045 every 8 connection(s)
I20260808 16:56:57.967170 16524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
Aug 08 16:56:57 dist-test-slave-1nl6 krb5kdc[16359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208217, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:57.976637 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 16524
I20260808 16:56:57.976684 16490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:57.967913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:39727 (local address 127.8.14.126:46859)
0808 16:56:57.967958 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:57.967960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:57.968183 (+ 223us) server_negotiation.cc:407] Connection header received
0808 16:56:57.969133 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:57.969134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:57.969144 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:57.969196 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:57.969789 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:57.970307 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:57.971532 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:57.971740 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:57.971805 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:57.973891 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:57.973892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:57.973892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:57.973899 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:57.975427 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:57.975820 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:57.975821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:57.975821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:57.975873 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:57.976078 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:57.976079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:57.976079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:57.976175 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:57.976230 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:57.976470 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 16:56:57.976574 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:56:57.976761 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:56:57.976761 16713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:57.967898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46859 (local address 127.8.14.65:39727)
0808 16:56:57.968046 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:57.968081 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:57.969041 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:57.969252 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:57.969255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:57.969523 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:57.969714 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:57.969719 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:57.970396 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:57.970397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:57.971422 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:57.971426 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:57.971902 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:57.971903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:57.972011 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:57.972448 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:57.972456 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:57.973809 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:57.975502 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:57.975509 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:57.975511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:57.975743 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:57.975930 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:57.975931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:57.975931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:57.976026 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:57.976235 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:57.976239 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:57.976343 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 16:56:57.976426 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 16:56:57.976556 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260808 16:56:57.976979 8249 external_mini_cluster.cc:1428] Setting key 966624c8088ffa9205eefab66dbcb4a0
I20260808 16:56:57.977346 16711 heartbeater.cc:347] Connected to a master server at 127.8.14.126:46859
I20260808 16:56:57.977442 16711 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:57.977636 16711 heartbeater.cc:511] Master 127.8.14.126:46859 requested a full tablet report, sending...
I20260808 16:56:57.978193 16431 ts_manager.cc:194] Registered new tserver with Master: 4bf9055714bd4252be25ae2ab06064da (127.8.14.65:35045)
I20260808 16:56:57.978788 16431 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:39727
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 16:56:58.002004 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46859
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:43583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:58.083557 16717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:58.083743 16717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:58.083765 16717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:58.084831 16717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:58.084865 16717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:58.084890 16717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:56:58.086344 16717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:43583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:58.086524 16717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:58.086694 16717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:58.091660 16730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:58.091686 16727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:58.091672 16734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:58.091718 16717 server_base.cc:1034] running on GCE node
I20260808 16:56:58.092082 16717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:58.092356 16717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:58.093524 16717 hybrid_clock.cc:648] HybridClock initialized: now 1786208218093506 us; error 37 us; skew 500 ppm
Aug 08 16:56:58 dist-test-slave-1nl6 krb5kdc[16359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208218, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:58.096139 16717 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:56:58.096621 16717 webserver.cc:460] Webserver started at http://127.8.14.66:43103/ using document root <none> and password file <none>
I20260808 16:56:58.096863 16717 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:58.096931 16717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:58.097088 16717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:58.098094 16717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "5dbf246714aa4796a0ee2e8b117ea6c8"
format_stamp: "Formatted at 2026-08-08 16:56:58 on dist-test-slave-1nl6"
server_key: "12b2bd1f345801ace31ab17a45a3f72c"
server_key_iv: "5acec777500f993fc212e829db4e02fb"
server_key_version: "encryptionkey@0"
I20260808 16:56:58.098460 16717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "5dbf246714aa4796a0ee2e8b117ea6c8"
format_stamp: "Formatted at 2026-08-08 16:56:58 on dist-test-slave-1nl6"
server_key: "12b2bd1f345801ace31ab17a45a3f72c"
server_key_iv: "5acec777500f993fc212e829db4e02fb"
server_key_version: "encryptionkey@0"
I20260808 16:56:58.099887 16717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:56:58.100626 16747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:58.100796 16717 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:58.100895 16717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "5dbf246714aa4796a0ee2e8b117ea6c8"
format_stamp: "Formatted at 2026-08-08 16:56:58 on dist-test-slave-1nl6"
server_key: "12b2bd1f345801ace31ab17a45a3f72c"
server_key_iv: "5acec777500f993fc212e829db4e02fb"
server_key_version: "encryptionkey@0"
I20260808 16:56:58.100975 16717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:58.110540 16717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:58.112321 16717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:58.112481 16717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:58.112689 16717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:58.112980 16717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:58.113034 16717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:58.113075 16717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:58.113116 16717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:58.118830 16717 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:39409
I20260808 16:56:58.118934 16912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:39409 every 8 connection(s)
I20260808 16:56:58.119287 16717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
Aug 08 16:56:58 dist-test-slave-1nl6 krb5kdc[16359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208218, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:58.126685 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 16717
I20260808 16:56:58.126770 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:56:58.126972 8249 external_mini_cluster.cc:1428] Setting key 389897351e722b86c9309b506f89dd06
I20260808 16:56:58.131647 16490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:58.120042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:32967 (local address 127.8.14.126:46859)
0808 16:56:58.120110 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:58.120112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:58.120331 (+ 219us) server_negotiation.cc:407] Connection header received
0808 16:56:58.121265 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:58.121267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:58.121279 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:58.121338 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:58.121938 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:58.122444 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:58.123503 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:58.123785 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:58.123871 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:58.126154 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:58.126155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:58.126156 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:58.126163 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:58.130174 (+ 4011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:58.130618 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:58.130619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:58.130619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:58.130684 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:58.130980 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:58.130981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:58.130981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:58.131101 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:58.131180 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:58.131459 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 16:56:58.131559 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 16:56:58.131816 16915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:58.120011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46859 (local address 127.8.14.66:32967)
0808 16:56:58.120194 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:58.120228 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:58.121147 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:58.121410 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:58.121413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:58.121658 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:58.121836 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:58.121840 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:58.122558 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:58.122559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:58.123407 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:58.123410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:58.123895 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:58.123896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:58.124096 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:58.124689 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:58.124702 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:58.126053 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:58.130274 (+ 4221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:58.130281 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:58.130284 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:58.130521 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:58.130759 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:58.130760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:58.130761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:58.130873 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:58.131171 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:58.131178 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:58.131323 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 16:56:58.131433 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 16:56:58.131588 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
I20260808 16:56:58.132321 16913 heartbeater.cc:347] Connected to a master server at 127.8.14.126:46859
I20260808 16:56:58.132431 16913 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:58.132634 16913 heartbeater.cc:511] Master 127.8.14.126:46859 requested a full tablet report, sending...
I20260808 16:56:58.133057 16431 ts_manager.cc:194] Registered new tserver with Master: 5dbf246714aa4796a0ee2e8b117ea6c8 (127.8.14.66:39409)
I20260808 16:56:58.133430 16431 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:32967
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 16:56:58.153309 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46859
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:43583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:58.233498 16920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:58.233716 16920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:58.233753 16920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:58.235030 16920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:58.235092 16920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:58.235157 16920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:56:58.236644 16920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:43583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:58.236903 16920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:58.237146 16920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:58.242106 16928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:58.242136 16926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:58.242321 16920 server_base.cc:1034] running on GCE node
W20260808 16:56:58.242107 16934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:58.242542 16920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:58.242796 16920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:58.243988 16920 hybrid_clock.cc:648] HybridClock initialized: now 1786208218243970 us; error 40 us; skew 500 ppm
Aug 08 16:56:58 dist-test-slave-1nl6 krb5kdc[16359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208218, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:58.246461 16920 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:56:58.246816 16920 webserver.cc:460] Webserver started at http://127.8.14.67:33621/ using document root <none> and password file <none>
I20260808 16:56:58.247067 16920 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:58.247113 16920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:58.247200 16920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:58.248152 16920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "00c50586866e47f98c19155ce4656fe7"
format_stamp: "Formatted at 2026-08-08 16:56:58 on dist-test-slave-1nl6"
server_key: "ced84ed4bcb1f63e89cf1616c9269d1d"
server_key_iv: "06d3e15661302702fe80471e12842c12"
server_key_version: "encryptionkey@0"
I20260808 16:56:58.248453 16920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "00c50586866e47f98c19155ce4656fe7"
format_stamp: "Formatted at 2026-08-08 16:56:58 on dist-test-slave-1nl6"
server_key: "ced84ed4bcb1f63e89cf1616c9269d1d"
server_key_iv: "06d3e15661302702fe80471e12842c12"
server_key_version: "encryptionkey@0"
I20260808 16:56:58.249663 16920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:58.250308 16949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:58.250479 16920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:58.250583 16920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "00c50586866e47f98c19155ce4656fe7"
format_stamp: "Formatted at 2026-08-08 16:56:58 on dist-test-slave-1nl6"
server_key: "ced84ed4bcb1f63e89cf1616c9269d1d"
server_key_iv: "06d3e15661302702fe80471e12842c12"
server_key_version: "encryptionkey@0"
I20260808 16:56:58.250661 16920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:58.264551 16920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:58.266409 16920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:58.266572 16920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:58.266799 16920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:56:58.267185 16920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:56:58.267244 16920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:58.267287 16920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:56:58.267320 16920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:58.273272 16920 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:32795
I20260808 16:56:58.273386 17117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:32795 every 8 connection(s)
I20260808 16:56:58.273674 16920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 16:56:58.277927 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 16920
I20260808 16:56:58.278046 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:56:58.278272 8249 external_mini_cluster.cc:1428] Setting key e4f264fe969bdc14a3e53c3ce30cb737
Aug 08 16:56:58 dist-test-slave-1nl6 krb5kdc[16359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208218, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:56:58.283074 16490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:58.274379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:54829 (local address 127.8.14.126:46859)
0808 16:56:58.274443 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:58.274445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:58.274660 (+ 215us) server_negotiation.cc:407] Connection header received
0808 16:56:58.275682 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:58.275683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:58.275696 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:58.275748 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:58.276345 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:58.276852 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:58.277835 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:58.278067 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:58.278141 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:58.280477 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:58.280478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:58.280479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:58.280487 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:58.281696 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:58.282089 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:58.282090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:58.282091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:58.282152 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:58.282417 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:58.282417 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:58.282418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:58.282535 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:58.282608 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:58.282881 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 16:56:58.282980 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 16:56:58.283267 17121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:58.274364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46859 (local address 127.8.14.67:54829)
0808 16:56:58.274543 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 16:56:58.274578 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 16:56:58.275562 (+ 984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:56:58.275820 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:56:58.275823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:56:58.276061 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:56:58.276238 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:58.276242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:58.276961 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:58.276962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:58.277735 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:56:58.277737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:56:58.278151 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:56:58.278152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:56:58.278276 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:58.278861 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:56:58.278870 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:56:58.280380 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:56:58.281785 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:58.281791 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:58.281792 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:58.282010 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:58.282231 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:56:58.282232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:56:58.282232 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:56:58.282348 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:56:58.282612 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:56:58.282619 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:56:58.282750 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 16:56:58.282859 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 16:56:58.283039 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260808 16:56:58.283732 17119 heartbeater.cc:347] Connected to a master server at 127.8.14.126:46859
I20260808 16:56:58.283826 17119 heartbeater.cc:464] Registering TS with master...
I20260808 16:56:58.283957 17119 heartbeater.cc:511] Master 127.8.14.126:46859 requested a full tablet report, sending...
I20260808 16:56:58.284281 16431 ts_manager.cc:194] Registered new tserver with Master: 00c50586866e47f98c19155ce4656fe7 (127.8.14.67:32795)
I20260808 16:56:58.284657 16431 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:54829
I20260808 16:56:58.290066 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/test-admin.keytab.
Aug 08 16:56:58 dist-test-slave-1nl6 krb5kdc[16359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:58.306681 8249 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 16:56:58.308291 17130 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:58.307573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33456 (local address 127.8.14.65:35045)
0808 16:56:58.307725 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:58.307733 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:58.307749 (+ 16us) server_negotiation.cc:407] Connection header received
0808 16:56:58.307774 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:58.307775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:58.307790 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:58.307861 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:58.308201 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33456: BlockingRecv error: recv got EOF from 127.0.0.1:33456 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
W20260808 16:56:58.308403 8249 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/krb5cc)
Aug 08 16:56:58 dist-test-slave-1nl6 krb5kdc[16359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:58.309417 8249 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 16:56:58 dist-test-slave-1nl6 krb5kdc[16359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:58.310322 8249 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 16:56:58 dist-test-slave-1nl6 krb5kdc[16359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.65@KRBTEST.COM
I20260808 16:56:58.318359 17130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:58.310847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33460 (local address 127.8.14.65:35045)
0808 16:56:58.310891 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:58.310893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:58.310937 (+ 44us) server_negotiation.cc:407] Connection header received
0808 16:56:58.311026 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:58.311027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:58.311036 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:58.311083 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:58.311447 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:58.312153 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:58.312811 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:58.313100 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:58.313151 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:58.315166 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:58.315175 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:58.315176 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:58.315183 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:58.316865 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:58.317170 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:58.317172 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:58.317172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:58.317265 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:58.317521 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:58.317522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:58.317522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:58.317752 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:58.317901 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:58.318152 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 16:56:58.318224 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 16:56:58.322986 17130 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:58.322306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33466 (local address 127.8.14.65:35045)
0808 16:56:58.322403 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:58.322405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:58.322414 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:56:58.322437 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:58.322438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:58.322450 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:58.322511 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:58.322930 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33466: BlockingRecv error: recv got EOF from 127.0.0.1:33466 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260808 16:56:58.323002 8249 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/krb5cc' of type FILE
Aug 08 16:56:58 dist-test-slave-1nl6 krb5kdc[16359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:58.324137 8249 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 16:56:58 dist-test-slave-1nl6 krb5kdc[16359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:58.325297 8249 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 16:56:58 dist-test-slave-1nl6 krb5kdc[16359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.65@KRBTEST.COM
I20260808 16:56:58.336388 17130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:58.328397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33478 (local address 127.8.14.65:35045)
0808 16:56:58.328438 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:58.328440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:58.328523 (+ 83us) server_negotiation.cc:407] Connection header received
0808 16:56:58.328584 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:58.328585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:58.328595 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:58.328646 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:58.329014 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:58.329638 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:58.330701 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:58.331032 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:58.331126 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:58.333514 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:58.333515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:58.333516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:58.333524 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:58.335252 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:58.335604 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:58.335605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:58.335605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:58.335677 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:58.335887 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:58.335888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:58.335889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:58.335991 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:58.336088 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:58.336220 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 16:56:58.336293 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 16:56:58.340354 17130 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:56:58.339706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33482 (local address 127.8.14.65:35045)
0808 16:56:58.339742 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:58.339744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:58.339775 (+ 31us) server_negotiation.cc:407] Connection header received
0808 16:56:58.339843 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:58.339844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:58.339851 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:58.339918 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:58.340322 (+ 404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33482: BlockingRecv error: recv got EOF from 127.0.0.1:33482 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 16:56:58.340605 8249 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786208190438830-8249-0/krb5kdc/krb5cc' of type FILE
Aug 08 16:56:58 dist-test-slave-1nl6 krb5kdc[16359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:58.341867 8249 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 16:56:58 dist-test-slave-1nl6 krb5kdc[16359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:58.342984 8249 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 16:56:58 dist-test-slave-1nl6 krb5kdc[16359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.65@KRBTEST.COM
I20260808 16:56:58.350749 17130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:58.343578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33490 (local address 127.8.14.65:35045)
0808 16:56:58.343616 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:58.343618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:58.343682 (+ 64us) server_negotiation.cc:407] Connection header received
0808 16:56:58.343739 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:58.343739 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:58.343748 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:58.343826 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:58.344253 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:58.345018 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:58.345823 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:58.346026 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:58.346111 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:58.348262 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:58.348263 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:58.348264 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:58.348271 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:58.349650 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:58.349993 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:58.349994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:58.349995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:58.350052 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:58.350263 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:58.350264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:58.350264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:58.350359 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:58.350452 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:58.350561 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 16:56:58.350642 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:56:58.351382 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 16524
I20260808 16:56:58.366544 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 16717
I20260808 16:56:58.392488 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 16920
I20260808 16:56:58.404906 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 16377
2026-08-08T16:56:58Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2065 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:56:58 dist-test-slave-1nl6 krb5kdc[17150](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:56:58 dist-test-slave-1nl6 krb5kdc[17150](info): set up 2 sockets
Aug 08 16:56:58 dist-test-slave-1nl6 krb5kdc[17150](info): commencing operation
krb5kdc: starting...
W20260808 16:56:59.625373 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:56:59 dist-test-slave-1nl6 krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:56:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:56:59Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.126.keytab.
Entry for principal oryx/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.126.keytab.
I20260808 16:56:59.741591 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:42179
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.126.keytab
--principal=oryx/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:42179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:56:59.822396 17168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:56:59.822592 17168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:56:59.822613 17168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:56:59.823732 17168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:56:59.823774 17168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:56:59.823810 17168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:56:59.823823 17168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:56:59.823835 17168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:56:59.825273 17168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:42179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.126.keytab
--principal=oryx/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:42179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:56:59.825464 17168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:56:59.825706 17168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:56:59.830952 17173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:59.830960 17179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:56:59.830962 17175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:56:59.831260 17168 server_base.cc:1034] running on GCE node
I20260808 16:56:59.831511 17168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:56:59.831789 17168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:56:59.832926 17168 hybrid_clock.cc:648] HybridClock initialized: now 1786208219832916 us; error 29 us; skew 500 ppm
Aug 08 16:56:59 dist-test-slave-1nl6 krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208219, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:56:59.838222 17168 init.cc:377] Logged in from keytab as oryx/127.8.14.126@KRBTEST.COM (short username oryx)
I20260808 16:56:59.838680 17168 webserver.cc:460] Webserver started at http://127.8.14.126:33845/ using document root <none> and password file <none>
I20260808 16:56:59.838874 17168 fs_manager.cc:359] Metadata directory not provided
I20260808 16:56:59.838984 17168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:56:59.839141 17168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:56:59.840161 17168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "1808f7e1cd1845c1a3da149aca6a718e"
format_stamp: "Formatted at 2026-08-08 16:56:59 on dist-test-slave-1nl6"
server_key: "2e4cfc347d65fe0c1bede7e0fcfe05f2"
server_key_iv: "1d48135bee7148005c05f6a5d0163053"
server_key_version: "encryptionkey@0"
I20260808 16:56:59.840524 17168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "1808f7e1cd1845c1a3da149aca6a718e"
format_stamp: "Formatted at 2026-08-08 16:56:59 on dist-test-slave-1nl6"
server_key: "2e4cfc347d65fe0c1bede7e0fcfe05f2"
server_key_iv: "1d48135bee7148005c05f6a5d0163053"
server_key_version: "encryptionkey@0"
I20260808 16:56:59.841889 17168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:56:59.842592 17189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:56:59.842784 17168 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:56:59.842862 17168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "1808f7e1cd1845c1a3da149aca6a718e"
format_stamp: "Formatted at 2026-08-08 16:56:59 on dist-test-slave-1nl6"
server_key: "2e4cfc347d65fe0c1bede7e0fcfe05f2"
server_key_iv: "1d48135bee7148005c05f6a5d0163053"
server_key_version: "encryptionkey@0"
I20260808 16:56:59.842969 17168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:56:59.875155 17168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:56:59.877177 17168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:56:59.877358 17168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:56:59.881392 17168 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:42179
I20260808 16:56:59.881445 17261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:42179 every 8 connection(s)
I20260808 16:56:59.881836 17168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:56:59.882818 17262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:56:59.885154 17262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e: Bootstrap starting.
I20260808 16:56:59.885936 17262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e: Neither blocks nor log segments found. Creating new log.
I20260808 16:56:59.886368 17262 log.cc:824] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e: Log is configured to *not* fsync() on all Append() calls
I20260808 16:56:59.886612 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 17168
I20260808 16:56:59.886732 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:56:59.886952 8249 external_mini_cluster.cc:1428] Setting key 0466d61e574fd42631c7cdcad6d42fd8
I20260808 16:56:59.887233 17262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e: No bootstrap required, opened a new log
I20260808 16:56:59.888989 17262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } }
I20260808 16:56:59.889148 17262 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:56:59.889204 17262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1808f7e1cd1845c1a3da149aca6a718e, State: Initialized, Role: FOLLOWER
I20260808 16:56:59.889338 17262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } }
I20260808 16:56:59.889441 17262 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:56:59.889487 17262 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:56:59.889535 17262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:56:59.890417 17262 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } }
I20260808 16:56:59.890540 17262 leader_election.cc:302] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1808f7e1cd1845c1a3da149aca6a718e; no voters:
I20260808 16:56:59.890714 17262 leader_election.cc:288] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:56:59.890748 17269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:56:59.890930 17269 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [term 1 LEADER]: Becoming Leader. State: Replica: 1808f7e1cd1845c1a3da149aca6a718e, State: Running, Role: LEADER
I20260808 16:56:59.891022 17269 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } }
I20260808 16:56:59.891407 17271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1808f7e1cd1845c1a3da149aca6a718e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } }
I20260808 16:56:59.891403 17270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1808f7e1cd1845c1a3da149aca6a718e. Latest consensus state: current_term: 1 leader_uuid: "1808f7e1cd1845c1a3da149aca6a718e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } }
I20260808 16:56:59.891522 17271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:59.891531 17270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [sys.catalog]: This master's current role is: LEADER
I20260808 16:56:59.891026 17262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:56:59.891721 17276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:56:59.892282 17276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:56:59.893909 17276 catalog_manager.cc:1347] Generated new cluster ID: b541f111632a432f91a657c16c22ed0f
I20260808 16:56:59.894799 17276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:56:59 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM
I20260808 16:56:59.898741 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:56:59.887481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59948 (local address 127.8.14.126:42179)
0808 16:56:59.887666 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 16:56:59.887676 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 16:56:59.887711 (+ 35us) server_negotiation.cc:407] Connection header received
0808 16:56:59.888975 (+ 1264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:56:59.888981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:56:59.889028 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:56:59.889172 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:56:59.889893 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:59.891002 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:59.892064 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:56:59.892648 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:56:59.892776 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:56:59.895795 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:56:59.895799 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:56:59.895799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:56:59.895809 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:56:59.897307 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:59.897682 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:59.897684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:59.897684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:59.897766 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:56:59.897943 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:56:59.897944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:56:59.897944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:56:59.898169 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:56:59.898274 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 16:56:59.898397 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 16:56:59.898496 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
I20260808 16:56:59.914007 17276 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:56:59.914755 17276 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:56:59.920014 17276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e: Generated new TSK 0
I20260808 16:56:59.920210 17276 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.65.keytab.
Entry for principal oryx/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.65.keytab.
I20260808 16:56:59.949092 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:42179
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.65.keytab
--principal=oryx/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:00.031335 17301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:00.031531 17301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:00.031554 17301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:00.032764 17301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:00.032814 17301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:00.032841 17301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:57:00.034365 17301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.65.keytab
--principal=oryx/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:42179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:00.034595 17301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:00.034842 17301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:00.039743 17311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:00.039743 17315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:00.039755 17312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:00.040050 17301 server_base.cc:1034] running on GCE node
I20260808 16:57:00.040262 17301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:00.040498 17301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:00.041639 17301 hybrid_clock.cc:648] HybridClock initialized: now 1786208220041625 us; error 36 us; skew 500 ppm
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208220, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:00.044199 17301 init.cc:377] Logged in from keytab as oryx/127.8.14.65@KRBTEST.COM (short username oryx)
I20260808 16:57:00.044677 17301 webserver.cc:460] Webserver started at http://127.8.14.65:37511/ using document root <none> and password file <none>
I20260808 16:57:00.044914 17301 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:00.044961 17301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:00.045114 17301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:00.046252 17301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "c196200dfad6458dbbe33229e9f06d7e"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "dbeb0fa300fb90748df76bdafcf91096"
server_key_iv: "f0e533cd10b126ace3303bcb99b2599c"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.046612 17301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "c196200dfad6458dbbe33229e9f06d7e"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "dbeb0fa300fb90748df76bdafcf91096"
server_key_iv: "f0e533cd10b126ace3303bcb99b2599c"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.048067 17301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:57:00.048823 17322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:00.049006 17301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:57:00.049098 17301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "c196200dfad6458dbbe33229e9f06d7e"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "dbeb0fa300fb90748df76bdafcf91096"
server_key_iv: "f0e533cd10b126ace3303bcb99b2599c"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.049156 17301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:00.073489 17301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:00.075533 17301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:00.075657 17301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:00.075884 17301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:00.076236 17301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:00.076273 17301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:00.076297 17301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:00.076349 17301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:00.082199 17301 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:43969
I20260808 16:57:00.082312 17507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:43969 every 8 connection(s)
I20260808 16:57:00.082638 17301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:57:00.084522 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 17301
I20260808 16:57:00.084604 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:57:00.084800 8249 external_mini_cluster.cc:1428] Setting key f1c125892ad1ba5ea7dd41f0d6d33abc
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208220, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM
I20260808 16:57:00.093561 17511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.083443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:35799)
0808 16:57:00.083612 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.083650 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.084191 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.084482 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.084485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.085203 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:00.085409 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.085416 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.086166 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.086168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.087058 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.087062 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.087485 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.087487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.087634 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.088221 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:00.088230 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:00.089879 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:00.091974 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.091981 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.091983 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.092239 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.092476 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.092477 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.092477 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.092677 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.093019 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:00.093024 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:00.093155 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.093245 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.093381 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260808 16:57:00.093555 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.083471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:35799 (local address 127.8.14.126:42179)
0808 16:57:00.083545 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.083547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.083745 (+ 198us) server_negotiation.cc:407] Connection header received
0808 16:57:00.084315 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.084316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.084335 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.084389 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.085526 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.086055 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.087164 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.087392 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.087476 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.089990 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.089992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.089992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.090006 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.091886 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.092323 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.092325 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.092325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.092397 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.092764 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.092765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.092766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.092945 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.093025 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.093300 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.093421 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 16:57:00.094378 17508 heartbeater.cc:347] Connected to a master server at 127.8.14.126:42179
I20260808 16:57:00.094502 17508 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:00.094735 17508 heartbeater.cc:511] Master 127.8.14.126:42179 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.8.14.66@KRBTEST.COM; defaulting to no policy
I20260808 16:57:00.095413 17209 ts_manager.cc:194] Registered new tserver with Master: c196200dfad6458dbbe33229e9f06d7e (127.8.14.65:43969)
I20260808 16:57:00.096071 17209 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:35799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.66.keytab.
Entry for principal oryx/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.66.keytab.
I20260808 16:57:00.113124 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:42179
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.66.keytab
--principal=oryx/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:00.199397 17516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:00.199645 17516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:00.199680 17516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:00.200783 17516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:00.200852 17516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:00.200894 17516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:57:00.202376 17516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.66.keytab
--principal=oryx/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:42179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:00.202679 17516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:00.202948 17516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:00.208009 17527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:00.208009 17525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:00.208060 17516 server_base.cc:1034] running on GCE node
W20260808 16:57:00.208014 17524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:00.208527 17516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:00.208806 17516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:00.209975 17516 hybrid_clock.cc:648] HybridClock initialized: now 1786208220209973 us; error 39 us; skew 500 ppm
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208220, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:00.212664 17516 init.cc:377] Logged in from keytab as oryx/127.8.14.66@KRBTEST.COM (short username oryx)
I20260808 16:57:00.213068 17516 webserver.cc:460] Webserver started at http://127.8.14.66:40695/ using document root <none> and password file <none>
I20260808 16:57:00.213263 17516 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:00.213313 17516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:00.213407 17516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:00.214442 17516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "cb7e31d9946142de9f8efa0adc06ab78"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "f7e49fb46116dd696260c09a003c8072"
server_key_iv: "444ed3c23250ef16c63780961c670057"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.214768 17516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "cb7e31d9946142de9f8efa0adc06ab78"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "f7e49fb46116dd696260c09a003c8072"
server_key_iv: "444ed3c23250ef16c63780961c670057"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.216159 17516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:57:00.216899 17536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:00.217056 17516 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:57:00.217152 17516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "cb7e31d9946142de9f8efa0adc06ab78"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "f7e49fb46116dd696260c09a003c8072"
server_key_iv: "444ed3c23250ef16c63780961c670057"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.217213 17516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:00.229689 17516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:00.231750 17516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:00.231876 17516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:00.232085 17516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:00.232446 17516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:00.232492 17516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:00.232515 17516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:00.232563 17516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:00.238469 17516 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:35569
I20260808 16:57:00.238551 17701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:35569 every 8 connection(s)
I20260808 16:57:00.238945 17516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:57:00.239142 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 17516
I20260808 16:57:00.239239 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:57:00.239466 8249 external_mini_cluster.cc:1428] Setting key ddceb59e4b3cf743484aeab02a16aa58
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208220, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM
I20260808 16:57:00.249455 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.239869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:51859 (local address 127.8.14.126:42179)
0808 16:57:00.239915 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.239917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.240187 (+ 270us) server_negotiation.cc:407] Connection header received
0808 16:57:00.240778 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.240779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.240793 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.240852 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.241824 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.242345 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.243654 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.243859 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.243914 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.246554 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.246555 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.246555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.246563 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.247968 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.248478 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.248478 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.248479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.248529 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.248815 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.248815 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.248815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.248913 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.248963 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.249321 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.249391 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:57:00.249724 17705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.239855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:51859)
0808 16:57:00.240028 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.240093 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.240656 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.240922 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.240926 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.241521 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:00.241712 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.241718 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.242452 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.242453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.243549 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.243553 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.243945 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.243946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.244101 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.244593 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:00.244605 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:00.246457 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:00.248057 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.248064 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.248066 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.248398 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.248600 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.248601 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.248601 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.248744 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.248982 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:00.248990 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:00.249166 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.249305 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.249495 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for oryx/127.8.14.67@KRBTEST.COM; defaulting to no policy
I20260808 16:57:00.250288 17703 heartbeater.cc:347] Connected to a master server at 127.8.14.126:42179
I20260808 16:57:00.250384 17703 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:00.250576 17703 heartbeater.cc:511] Master 127.8.14.126:42179 requested a full tablet report, sending...
I20260808 16:57:00.251037 17209 ts_manager.cc:194] Registered new tserver with Master: cb7e31d9946142de9f8efa0adc06ab78 (127.8.14.66:35569)
I20260808 16:57:00.251434 17209 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:51859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.67.keytab.
Entry for principal oryx/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.67.keytab.
I20260808 16:57:00.265551 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:42179
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.67.keytab
--principal=oryx/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:00.347121 17709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:00.347342 17709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:00.347383 17709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:00.348506 17709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:00.348574 17709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:00.348619 17709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:57:00.350073 17709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.67.keytab
--principal=oryx/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:42179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:00.350337 17709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:00.350565 17709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:00.355588 17720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:00.355611 17722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:00.355608 17725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:00.355849 17709 server_base.cc:1034] running on GCE node
I20260808 16:57:00.356065 17709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:00.356308 17709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:00.357448 17709 hybrid_clock.cc:648] HybridClock initialized: now 1786208220357443 us; error 30 us; skew 500 ppm
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208220, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:00.360481 17709 init.cc:377] Logged in from keytab as oryx/127.8.14.67@KRBTEST.COM (short username oryx)
I20260808 16:57:00.360906 17709 webserver.cc:460] Webserver started at http://127.8.14.67:34775/ using document root <none> and password file <none>
I20260808 16:57:00.361127 17709 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:00.361184 17709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:00.361364 17709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:00.362376 17709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "ae940e53ec88472ba06e15f8e4b95748"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "e03eabb1529605a9a50a61a0eaf918ea"
server_key_iv: "9a0f0cdfa91d9795586f5947ee54171a"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.362741 17709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "ae940e53ec88472ba06e15f8e4b95748"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "e03eabb1529605a9a50a61a0eaf918ea"
server_key_iv: "9a0f0cdfa91d9795586f5947ee54171a"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.364256 17709 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:57:00.364975 17739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:00.365170 17709 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:57:00.365267 17709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "ae940e53ec88472ba06e15f8e4b95748"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "e03eabb1529605a9a50a61a0eaf918ea"
server_key_iv: "9a0f0cdfa91d9795586f5947ee54171a"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.365345 17709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:00.379417 17709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:00.381717 17709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:00.381829 17709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:00.382071 17709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:00.382401 17709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:00.382457 17709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:00.382490 17709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:00.382519 17709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:00.389400 17709 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:33471
I20260808 16:57:00.389439 17909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:33471 every 8 connection(s)
I20260808 16:57:00.389784 17709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 16:57:00.390205 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 17709
I20260808 16:57:00.390305 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:57:00.390511 8249 external_mini_cluster.cc:1428] Setting key ca14819b78bc2f838f204b8ac0d332c0
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208220, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM
I20260808 16:57:00.399446 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.390574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:35193 (local address 127.8.14.126:42179)
0808 16:57:00.390647 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.390649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.390913 (+ 264us) server_negotiation.cc:407] Connection header received
0808 16:57:00.391617 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.391618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.391631 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.391702 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.392743 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.393357 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.394352 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.394548 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.394619 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.396859 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.396860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.396860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.396867 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.398056 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.398438 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.398439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.398440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.398489 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.398823 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.398823 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.398824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.399000 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.399077 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.399314 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.399379 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 16:57:00.399593 17912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.390540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:35193)
0808 16:57:00.390758 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.390807 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.391518 (+ 711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.391763 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.391767 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.392456 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:00.392648 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.392653 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.393481 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.393482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.394246 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.394248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.394626 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.394627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.394795 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.395390 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:00.395399 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:00.396764 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:00.398125 (+ 1361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.398131 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.398132 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.398354 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.398558 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.398559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.398559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.398655 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.399068 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:00.399073 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:00.399189 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.399277 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.399397 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":118,"threads_started":1}
I20260808 16:57:00.400082 17910 heartbeater.cc:347] Connected to a master server at 127.8.14.126:42179
I20260808 16:57:00.400199 17910 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:00.400377 17910 heartbeater.cc:511] Master 127.8.14.126:42179 requested a full tablet report, sending...
I20260808 16:57:00.400763 17209 ts_manager.cc:194] Registered new tserver with Master: ae940e53ec88472ba06e15f8e4b95748 (127.8.14.67:33471)
I20260808 16:57:00.401122 17209 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:35193
I20260808 16:57:00.402273 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.14.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.14.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.14.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.125.keytab.
Entry for principal oryx/127.8.14.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.125.keytab.
I20260808 16:57:00.430321 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.125:35539
--webserver_interface=127.8.14.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.125.keytab
--principal=oryx/127.8.14.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:42179,127.8.14.125:35539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:00.514319 17918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:00.514541 17918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:00.514580 17918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:00.515769 17918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:57:00.515841 17918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:00.515866 17918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:00.515895 17918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:57:00.515928 17918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:57:00.517359 17918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:42179,127.8.14.125:35539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.125.keytab
--principal=oryx/127.8.14.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.125:35539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.14.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
W20260808 16:57:00.517501 17918 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.14.126:42179,127.8.14.125:35539'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 16:57:00.517654 17918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:00.517882 17918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:00.522701 17927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:00.522692 17929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:00.522691 17926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:00.522891 17918 server_base.cc:1034] running on GCE node
I20260808 16:57:00.523149 17918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:00.523434 17918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:00.524578 17918 hybrid_clock.cc:648] HybridClock initialized: now 1786208220524569 us; error 29 us; skew 500 ppm
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208220, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:00.527168 17918 init.cc:377] Logged in from keytab as oryx/127.8.14.125@KRBTEST.COM (short username oryx)
I20260808 16:57:00.527593 17918 webserver.cc:460] Webserver started at http://127.8.14.125:40267/ using document root <none> and password file <none>
I20260808 16:57:00.527838 17918 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:00.527907 17918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:00.528065 17918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:00.529066 17918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/data/instance:
uuid: "a2cc6282c44644d4b7a4f199a38d6bc8"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "ae51cce9504bd6a6c3aaed0c823c915c"
server_key_iv: "0bc43af1617b481c02150158e4bd3006"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.529429 17918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/wal/instance:
uuid: "a2cc6282c44644d4b7a4f199a38d6bc8"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "ae51cce9504bd6a6c3aaed0c823c915c"
server_key_iv: "0bc43af1617b481c02150158e4bd3006"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.530704 17918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:57:00.531497 17944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:00.531665 17918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:00.531771 17918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/wal
uuid: "a2cc6282c44644d4b7a4f199a38d6bc8"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "ae51cce9504bd6a6c3aaed0c823c915c"
server_key_iv: "0bc43af1617b481c02150158e4bd3006"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.531855 17918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:00.541558 17918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:00.543886 17918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:00.544041 17918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:00.548053 17918 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.125:35539
I20260808 16:57:00.548085 18017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.125:35539 every 8 connection(s)
I20260808 16:57:00.548416 17918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/data/info.pb
I20260808 16:57:00.549403 18018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:00.550863 18018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:57:00.555985 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 17918
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208220, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.125@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM
I20260808 16:57:00.560243 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.551576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59956 (local address 127.8.14.126:42179)
0808 16:57:00.551631 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.551633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.551847 (+ 214us) server_negotiation.cc:407] Connection header received
0808 16:57:00.552395 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.552396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.552407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.552460 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.553118 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.553599 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.554527 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.554706 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.554755 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.557187 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.557188 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.557189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.557195 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.558751 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.559251 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.559252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.559252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.559307 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.559574 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.559574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.559575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.559715 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.559769 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.560057 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.560140 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:57:00.560300 18020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.551580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:59956)
0808 16:57:00.551729 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.551760 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.552307 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.552509 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.552512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.552861 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:00.553039 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.553044 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.553676 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.553676 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.554455 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.554457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.554789 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.554790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.554890 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.555366 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:00.555374 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:00.557105 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:00.558833 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.558839 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.558841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.559178 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.559376 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.559377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.559378 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.559509 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.559786 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:00.559791 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:00.559930 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.560012 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.560122 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260808 16:57:00.560891 18018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.14.125@KRBTEST.COM
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208220, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.125@KRBTEST.COM for oryx/127.8.14.125@KRBTEST.COM
I20260808 16:57:00.565881 18022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.556655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35678 (local address 127.8.14.125:35539)
0808 16:57:00.556792 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.556795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.556885 (+ 90us) server_negotiation.cc:407] Connection header received
0808 16:57:00.556929 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.556931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.556948 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.557009 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.557700 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.558799 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.559824 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.560206 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.560279 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.562812 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.562820 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.562820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.562827 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.564572 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.564972 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.564975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.564975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.565054 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.565258 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.565259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.565259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.565412 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.565581 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.565662 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.565747 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260808 16:57:00.568202 18020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.561115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:35539 (local address 127.0.0.1:35684)
0808 16:57:00.561163 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.561172 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.561210 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.561494 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.561495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.561629 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:00.561668 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.561669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.562320 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.562321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.562839 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.562840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.563352 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.563353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.563435 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.563723 (+ 288us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:00.563727 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:00.564988 (+ 1261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:00.567310 (+ 2322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.567311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.567311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.567471 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.567641 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.567642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.567642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.567691 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.567895 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:00.567896 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:00.567934 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.568003 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.568095 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 16:57:00.568290 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.561223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35684 (local address 127.8.14.125:35539)
0808 16:57:00.561390 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.561392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.561397 (+ 5us) server_negotiation.cc:407] Connection header received
0808 16:57:00.561411 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.561412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.561418 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.561493 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.561749 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.562228 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.562941 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.563244 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.563313 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.565074 (+ 1761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.565075 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.565075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.565081 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.567235 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.567533 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.567534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.567534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.567584 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.567728 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.567728 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.567728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.567809 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.567890 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.568023 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.568144 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":152,"thread_start_us":96,"threads_started":1}
I20260808 16:57:00.569510 18018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: Bootstrap starting.
I20260808 16:57:00.570047 18018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:00.570266 18018 log.cc:824] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: Log is configured to *not* fsync() on all Append() calls
I20260808 16:57:00.570959 18018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: No bootstrap required, opened a new log
I20260808 16:57:00.572258 18018 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } }
I20260808 16:57:00.572373 18018 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:00.572396 18018 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2cc6282c44644d4b7a4f199a38d6bc8, State: Initialized, Role: FOLLOWER
I20260808 16:57:00.572480 18018 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } }
I20260808 16:57:00.572698 18028 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } } }
I20260808 16:57:00.572768 18018 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:57:00.572783 18028 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 16:57:00.574348 18046 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:57:00.574407 18046 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:57:00.576095 17918 master_runner.cc:428] Detected that this master a2cc6282c44644d4b7a4f199a38d6bc8 is joining an existing cluster
I20260808 16:57:00.576180 17918 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.14.125:35539
I20260808 16:57:00.584013 18055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.577591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:35539 (local address 127.0.0.1:35698)
0808 16:57:00.577706 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.577714 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.577762 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.577891 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.577892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.578066 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:00.578140 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.578142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.579011 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.579011 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.579756 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.579758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.580183 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.580184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.580239 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.580779 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:00.580784 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:00.581176 (+ 392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:00.582603 (+ 1427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.582604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.582605 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.582768 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.583007 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.583008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.583009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.583064 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.583634 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:00.583635 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:00.583679 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.583752 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.583867 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":78,"threads_started":1}
I20260808 16:57:00.584030 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.577553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35698 (local address 127.8.14.125:35539)
0808 16:57:00.577582 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.577584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.577751 (+ 167us) server_negotiation.cc:407] Connection header received
0808 16:57:00.577824 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.577824 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.577833 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.577886 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.578221 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.578943 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.579849 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.580054 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.580140 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.581242 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.581243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.581244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.581250 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.582525 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.582851 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.582852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.582852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.582940 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.583136 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.583137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.583138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.583552 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.583628 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.583797 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.583929 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":16}
I20260808 16:57:00.584723 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.577295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:59964)
0808 16:57:00.577471 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.577482 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.577543 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.577769 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.577770 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.577970 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:00.578050 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.578052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.578717 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.578718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.579601 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.579603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.580106 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.580107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.580173 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.580604 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:00.580608 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:00.581072 (+ 464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:00.583418 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.583419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.583419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.583626 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.583847 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.583848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.583848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.583906 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.584395 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:00.584396 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:00.584437 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.584520 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.584622 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":82,"threads_started":1}
I20260808 16:57:00.584739 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.577349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59964 (local address 127.8.14.126:42179)
0808 16:57:00.577397 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.577399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.577548 (+ 149us) server_negotiation.cc:407] Connection header received
0808 16:57:00.577648 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.577649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.577660 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.577721 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.578141 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.578632 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.579689 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.580014 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.580093 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.581167 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.581169 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.581169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.581177 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.583322 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.583698 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.583699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.583699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.583764 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.583983 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.583983 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.583984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.584123 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.584168 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.584565 (+ 397us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.584641 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208219, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.126@KRBTEST.COM for oryx/127.8.14.125@KRBTEST.COM
I20260808 16:57:00.594389 18057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.586894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:35539 (local address 127.0.0.1:35708)
0808 16:57:00.587043 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.587087 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.587159 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.587293 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.587295 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.587531 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:00.587571 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.587573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.588472 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.588473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.589020 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.589021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.589463 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.589464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.589518 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.589984 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:00.589989 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:00.591524 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:00.593268 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.593275 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.593277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.593476 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.593679 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.593679 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.593679 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.593733 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.594032 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:00.594040 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:00.594101 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.594204 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.594305 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":100,"threads_started":1}
I20260808 16:57:00.594424 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.586958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35708 (local address 127.8.14.125:35539)
0808 16:57:00.586995 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.586996 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.587153 (+ 157us) server_negotiation.cc:407] Connection header received
0808 16:57:00.587214 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.587215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.587224 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.587291 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.587641 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.588388 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.589096 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.589386 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.589456 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.591611 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.591612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.591613 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.591620 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.593178 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.593549 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.593550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.593550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.593609 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.593800 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.593801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.593802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.593929 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.593997 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.594258 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.594336 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:57:00.595033 17209 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.14.125:35539: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8"
member_type: NON_VOTER
last_known_addr {
host: "127.8.14.125"
port: 35539
}
attrs {
promote: true
}
}
dest_uuid: "1808f7e1cd1845c1a3da149aca6a718e"
cas_config_opid_index: -1
I20260808 16:57:00.595418 17209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: NON_VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: true } }
I20260808 16:57:00.595923 18060 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a2cc6282c44644d4b7a4f199a38d6bc8 (127.8.14.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: NON_VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: true } } }
I20260808 16:57:00.596195 17981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:00.596846 18060 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.14.125:35539
I20260808 16:57:00.597102 17981 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Refusing update from remote peer 1808f7e1cd1845c1a3da149aca6a718e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 16:57:00.597151 17918 master.cc:537] Master@127.8.14.125:35539 shutting down...
I20260808 16:57:00.597271 18058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1808f7e1cd1845c1a3da149aca6a718e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: NON_VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: true } } }
I20260808 16:57:00.597365 18058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [sys.catalog]: This master's current role is: LEADER
I20260808 16:57:00.597468 18058 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: NON_VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 16:57:00.598178 17195 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 16:57:00.598753 17195 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e -> Peer a2cc6282c44644d4b7a4f199a38d6bc8 (127.8.14.125:35539): Couldn't send request to peer a2cc6282c44644d4b7a4f199a38d6bc8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 16:57:00.600047 17918 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:57:00.600152 17918 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:00.600208 17918 tablet_replica.cc:330] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: stopping tablet replica
I20260808 16:57:00.612748 17918 master.cc:552] Master@127.8.14.125:35539 shutdown complete.
I20260808 16:57:00.612825 17918 master_runner.cc:305] Clearing existing system tablet
I20260808 16:57:00.612954 17918 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:00.613019 17918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:00.613636 17918 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260808 16:57:00.614145 18072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:00.614275 17918 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:00.614332 17918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:00.614377 17918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/wal
uuid: "a2cc6282c44644d4b7a4f199a38d6bc8"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "ae51cce9504bd6a6c3aaed0c823c915c"
server_key_iv: "0bc43af1617b481c02150158e4bd3006"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.615103 17918 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:00.616436 17918 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 16:57:00.616494 17918 log.cc:1196] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 16:57:00.616772 17918 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: Deleting consensus metadata
I20260808 16:57:00.616937 17918 master_runner.cc:315] Copying system tablet from 127.8.14.126:42179
I20260808 16:57:00.617579 17918 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: tablet copy: Beginning tablet copy session from remote peer at address 127.8.14.126:42179
I20260808 16:57:00.624096 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.618003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59972 (local address 127.8.14.126:42179)
0808 16:57:00.618047 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.618049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.618088 (+ 39us) server_negotiation.cc:407] Connection header received
0808 16:57:00.618165 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.618165 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.618174 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.618221 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.618567 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.619145 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.620232 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.620481 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.620536 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.621536 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.621537 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.621537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.621543 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.623046 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.623424 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.623424 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.623425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.623477 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.623666 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.623667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.623667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.623764 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.623817 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.623978 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.624035 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:57:00.624177 18079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.617902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:59972)
0808 16:57:00.618027 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.618039 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.618103 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.618227 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.618228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.618427 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:00.618496 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.618498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.619218 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.619219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.620152 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.620155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.620551 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.620552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.620624 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.621046 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:00.621052 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:00.621447 (+ 395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:00.623120 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.623121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.623122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.623345 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.623536 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.623537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.623538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.623605 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.623826 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:00.623827 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:00.623874 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.623962 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.624065 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":85,"threads_started":1}
I20260808 16:57:00.624699 17241 tablet_copy_service.cc:140] P 1808f7e1cd1845c1a3da149aca6a718e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a2cc6282c44644d4b7a4f199a38d6bc8 ({username='oryx', principal='oryx/127.8.14.125@KRBTEST.COM'} at 127.0.0.1:59972)
I20260808 16:57:00.624861 17241 tablet_copy_service.cc:161] P 1808f7e1cd1845c1a3da149aca6a718e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a2cc6282c44644d4b7a4f199a38d6bc8 at {username='oryx', principal='oryx/127.8.14.125@KRBTEST.COM'} at 127.0.0.1:59972: session id = a2cc6282c44644d4b7a4f199a38d6bc8-00000000000000000000000000000000
I20260808 16:57:00.625593 17241 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 16:57:00.626194 17918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 16:57:00.627277 17918 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: tablet copy: Starting download of 0 data blocks...
I20260808 16:57:00.627446 17918 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: tablet copy: Starting download of 1 WAL segments...
I20260808 16:57:00.628418 17918 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 16:57:00.629253 17241 tablet_copy_service.cc:342] P 1808f7e1cd1845c1a3da149aca6a718e: Request end of tablet copy session a2cc6282c44644d4b7a4f199a38d6bc8-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.14.125@KRBTEST.COM'} at 127.0.0.1:59972
I20260808 16:57:00.629319 17241 tablet_copy_service.cc:433] P 1808f7e1cd1845c1a3da149aca6a718e: ending tablet copy session a2cc6282c44644d4b7a4f199a38d6bc8-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a2cc6282c44644d4b7a4f199a38d6bc8
I20260808 16:57:00.631927 17918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:00.632963 18083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:00.632979 18080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:00.633101 18081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:00.633158 17918 server_base.cc:1034] running on GCE node
I20260808 16:57:00.633337 17918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:00.633487 17918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:00.634608 17918 hybrid_clock.cc:648] HybridClock initialized: now 1786208220634596 us; error 27 us; skew 500 ppm
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208220, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:00.636163 17918 init.cc:377] Logged in from keytab as oryx/127.8.14.125@KRBTEST.COM (short username oryx)
I20260808 16:57:00.636464 17918 webserver.cc:460] Webserver started at http://127.8.14.125:44031/ using document root <none> and password file <none>
I20260808 16:57:00.636590 17918 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:00.636639 17918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:00.637336 17918 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:57:00.637871 18093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:00.638029 17918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:57:00.638089 17918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/wal
uuid: "a2cc6282c44644d4b7a4f199a38d6bc8"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "ae51cce9504bd6a6c3aaed0c823c915c"
server_key_iv: "0bc43af1617b481c02150158e4bd3006"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.638150 17918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:00.651968 17918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:00.653594 17918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:00.656692 17918 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.125:35539
I20260808 16:57:00.656991 18171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.125:35539 every 8 connection(s)
I20260808 16:57:00.657109 17918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-1/data/info.pb
I20260808 16:57:00.657805 18172 sys_catalog.cc:263] Verifying existing consensus state
I20260808 16:57:00.658229 18172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: Bootstrap starting.
I20260808 16:57:00.660516 18172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:57:00.660743 18172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: Bootstrap complete.
I20260808 16:57:00.660944 18172 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: NON_VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: true } }
I20260808 16:57:00.661036 18172 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a2cc6282c44644d4b7a4f199a38d6bc8, State: Initialized, Role: LEARNER
I20260808 16:57:00.661087 18172 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: NON_VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: true } }
I20260808 16:57:00.661298 18172 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:57:00.661280 18176 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: NON_VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: true } } }
I20260808 16:57:00.661370 18176 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [sys.catalog]: This master's current role is: LEARNER
I20260808 16:57:00.662822 18193 catalog_manager.cc:1259] Loaded cluster ID: b541f111632a432f91a657c16c22ed0f
I20260808 16:57:00.662894 18193 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: loading cluster ID for follower catalog manager: success
I20260808 16:57:00.663916 18193 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: acquiring CA information for follower catalog manager: success
I20260808 16:57:00.664227 18193 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 16:57:00.673942 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.667466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35720 (local address 127.8.14.125:35539)
0808 16:57:00.667589 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.667591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.667599 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:57:00.667628 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.667629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.667638 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.667711 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.668291 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.668924 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.669692 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.669905 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.669983 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.671251 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.671252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.671253 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.671259 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.672726 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.673116 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.673117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.673118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.673180 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.673388 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.673389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.673389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.673495 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.673612 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.673711 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.673795 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
WARNING: no policy specified for oryx/127.8.14.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.14.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.14.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.124.keytab.
Entry for principal oryx/127.8.14.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.124.keytab.
I20260808 16:57:00.701719 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.124:46491
--webserver_interface=127.8.14.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.124.keytab
--principal=oryx/127.8.14.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:42179,127.8.14.125:35539,127.8.14.124:46491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:00.783969 18197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:00.784164 18197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:00.784184 18197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:00.785322 18197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:57:00.785364 18197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:00.785378 18197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:00.785391 18197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:57:00.785442 18197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:57:00.786962 18197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:42179,127.8.14.125:35539,127.8.14.124:46491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.124.keytab
--principal=oryx/127.8.14.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.124:46491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.14.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:00.787227 18197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:00.787463 18197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:00.792232 18205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:00.792245 18207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:00.792292 18204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:00.792532 18197 server_base.cc:1034] running on GCE node
I20260808 16:57:00.792815 18197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:00.793089 18197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:00.794271 18197 hybrid_clock.cc:648] HybridClock initialized: now 1786208220794260 us; error 41 us; skew 500 ppm
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208220, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:00.796856 18197 init.cc:377] Logged in from keytab as oryx/127.8.14.124@KRBTEST.COM (short username oryx)
I20260808 16:57:00.797317 18197 webserver.cc:460] Webserver started at http://127.8.14.124:45953/ using document root <none> and password file <none>
I20260808 16:57:00.797551 18197 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:00.797619 18197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:00.797766 18197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:00.798794 18197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/data/instance:
uuid: "abd97edeb20a4efd922778124c11f19b"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "996c248681956f8fc38e316e6126f6e3"
server_key_iv: "58c4543a5484e993d80e7db022818ae5"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.799264 18197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/wal/instance:
uuid: "abd97edeb20a4efd922778124c11f19b"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "996c248681956f8fc38e316e6126f6e3"
server_key_iv: "58c4543a5484e993d80e7db022818ae5"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.800590 18197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:57:00.801257 18218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:00.801425 18197 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:57:00.801532 18197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/wal
uuid: "abd97edeb20a4efd922778124c11f19b"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "996c248681956f8fc38e316e6126f6e3"
server_key_iv: "58c4543a5484e993d80e7db022818ae5"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.801604 18197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:00.812331 18197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:00.814661 18197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:00.814831 18197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:00.819051 18197 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.124:46491
I20260808 16:57:00.819083 18292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.124:46491 every 8 connection(s)
I20260808 16:57:00.819428 18197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/data/info.pb
I20260808 16:57:00.820456 18293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:00.821969 18293 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:57:00.826668 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 18197
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208220, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.124@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM
I20260808 16:57:00.830950 18297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.822533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:59974)
0808 16:57:00.822699 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.822739 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.823269 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.823473 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.823476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.823710 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:00.823897 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.823901 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.824513 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.824513 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.825282 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.825283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.825593 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.825594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.825687 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.826206 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:00.826215 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:00.827858 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:00.829479 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.829486 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.829488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.829709 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.829951 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.829952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.829952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.830091 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.830415 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:00.830422 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:00.830560 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.830643 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.830749 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260808 16:57:00.830953 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.822694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59974 (local address 127.8.14.126:42179)
0808 16:57:00.822749 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.822751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.822820 (+ 69us) server_negotiation.cc:407] Connection header received
0808 16:57:00.823356 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.823357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.823366 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.823423 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.823963 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.824437 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.825352 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.825522 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.825570 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.827949 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.827951 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.827951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.827959 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.829375 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.829807 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.829808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.829809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.829870 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.830158 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.830159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.830159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.830320 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.830385 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.830705 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.830794 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:57:00.831611 18293 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.14.124@KRBTEST.COM
I20260808 16:57:00.835573 18298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.827247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56818 (local address 127.8.14.124:46491)
0808 16:57:00.827382 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.827387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.827467 (+ 80us) server_negotiation.cc:407] Connection header received
0808 16:57:00.827536 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.827539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.827568 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.827655 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.828218 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.828870 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.829590 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.829889 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.829985 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.832851 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.832857 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.832857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.832863 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.834435 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.834792 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.834796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.834796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.834860 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.835089 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.835090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.835090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.835213 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.835317 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.835450 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.835509 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208220, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.124@KRBTEST.COM for oryx/127.8.14.125@KRBTEST.COM
I20260808 16:57:00.838217 18297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.831861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:35539 (local address 127.0.0.1:35724)
0808 16:57:00.831914 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.831925 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.831992 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.832146 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.832146 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.832314 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:00.832375 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.832377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.833211 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.833212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.833781 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.833783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.834162 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.834163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.834239 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.834612 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:00.834616 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:00.835922 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:00.837356 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.837357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.837357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.837517 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.837682 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.837683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.837683 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.837733 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.837952 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:00.837953 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:00.838010 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.838093 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.838161 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 16:57:00.838224 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.831920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35724 (local address 127.8.14.125:35539)
0808 16:57:00.831977 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.831979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.831989 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:57:00.832071 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.832071 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.832081 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.832131 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.832474 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.833102 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.833885 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.834089 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.834164 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.836016 (+ 1852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.836018 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.836018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.836025 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.837282 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.837579 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.837579 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.837580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.837626 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.837787 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.837788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.837788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.837897 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.837946 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.838103 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.838166 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 16:57:00.838666 18293 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46491 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208220, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.124@KRBTEST.COM for oryx/127.8.14.124@KRBTEST.COM
I20260808 16:57:00.844118 18297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.838937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46491 (local address 127.0.0.1:56822)
0808 16:57:00.838971 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.838982 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.839038 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.839189 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.839190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.839304 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:00.839331 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.839332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.839855 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.839856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.840399 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.840401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.840706 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.840707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.840741 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.841049 (+ 308us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:00.841053 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:00.842068 (+ 1015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:00.843389 (+ 1321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.843389 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.843390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.843545 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.843705 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.843706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.843706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.843753 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.843921 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:00.843921 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:00.843951 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.843999 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.844067 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 16:57:00.844177 18298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.838966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56822 (local address 127.8.14.124:46491)
0808 16:57:00.838996 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.838997 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.839026 (+ 29us) server_negotiation.cc:407] Connection header received
0808 16:57:00.839112 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.839113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.839122 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.839184 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.839373 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.839795 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.840466 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.840644 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.840695 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.842141 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.842142 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.842142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.842148 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.843319 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.843606 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.843606 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.843607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.843651 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.843797 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.843797 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.843797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.843874 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.843915 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.844012 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.844108 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":14}
I20260808 16:57:00.845458 18293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: Bootstrap starting.
I20260808 16:57:00.846289 18293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:00.846586 18293 log.cc:824] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: Log is configured to *not* fsync() on all Append() calls
I20260808 16:57:00.847316 18293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: No bootstrap required, opened a new log
I20260808 16:57:00.848699 18293 raft_consensus.cc:348] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46491 } }
I20260808 16:57:00.848821 18293 raft_consensus.cc:374] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:00.848868 18293 raft_consensus.cc:729] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abd97edeb20a4efd922778124c11f19b, State: Initialized, Role: FOLLOWER
I20260808 16:57:00.848996 18293 consensus_queue.cc:260] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46491 } }
I20260808 16:57:00.849306 18293 sys_catalog.cc:564] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:57:00.849278 18300 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46491 } } }
I20260808 16:57:00.849387 18300 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b [sys.catalog]: This master's current role is: FOLLOWER
W20260808 16:57:00.851462 18315 catalog_manager.cc:1558] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:57:00.851521 18315 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:57:00.853899 18197 master_runner.cc:428] Detected that this master abd97edeb20a4efd922778124c11f19b is joining an existing cluster
I20260808 16:57:00.853945 18197 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.14.124:46491
I20260808 16:57:00.878974 18298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.855056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56832 (local address 127.8.14.124:46491)
0808 16:57:00.855099 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.855101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.855890 (+ 789us) server_negotiation.cc:407] Connection header received
0808 16:57:00.855953 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.855954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.855966 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.856044 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.856428 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.857134 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.858349 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.858662 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.858705 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.859873 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.859875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.859875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.859881 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.877513 (+ 17632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.878086 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.878087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.878087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.878136 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.878345 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.878346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.878346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.878506 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.878547 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.878785 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.878866 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:57:00.879014 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.855902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35738 (local address 127.8.14.125:35539)
0808 16:57:00.855945 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.855946 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.861464 (+ 5518us) server_negotiation.cc:407] Connection header received
0808 16:57:00.861531 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.861532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.861541 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.861634 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.861964 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.862649 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.863781 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.864072 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.864146 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.875734 (+ 11588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.875736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.875736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.875743 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.877482 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.878047 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.878048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.878049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.878109 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.878345 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.878346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.878346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.878479 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.878534 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.878794 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.878888 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:57:00.879151 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.855217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59988 (local address 127.8.14.126:42179)
0808 16:57:00.855257 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.855259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.856775 (+ 1516us) server_negotiation.cc:407] Connection header received
0808 16:57:00.856880 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.856881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.856891 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.856920 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.857347 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.858080 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.859187 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.859535 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.859609 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.875703 (+ 16094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.875705 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.875706 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.875714 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.877564 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.878108 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.878109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.878109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.878164 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.878370 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.878371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.878371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.878492 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.878534 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.878961 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.879041 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:57:00.879529 18322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.855991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:35539 (local address 127.0.0.1:35738)
0808 16:57:00.861393 (+ 5402us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.861405 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.861475 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.861643 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.861644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.861829 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:00.861884 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.861886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.862759 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.862760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.863691 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.863694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.864155 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.864155 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.864223 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.865446 (+ 1223us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:00.865451 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:00.875635 (+ 10184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:00.877584 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.877585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.877585 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.877954 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.878181 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.878182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.878182 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.878254 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.878572 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:00.878573 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:00.878671 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.879248 (+ 577us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.879389 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5383,"mutex_wait_us":1,"thread_start_us":81,"threads_started":1}
I20260808 16:57:00.879717 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.854970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46491 (local address 127.0.0.1:56832)
0808 16:57:00.855827 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.855837 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.855900 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.856075 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.856076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.856276 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:00.856364 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.856366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.857234 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.857235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.858238 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.858241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.858791 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.858792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.858866 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.859364 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:00.859369 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:00.859767 (+ 398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:00.877609 (+ 17842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.877609 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.877610 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.878007 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.878198 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.878198 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.878199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.878270 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.878588 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:00.878588 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:00.878686 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.879097 (+ 411us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.879208 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":48,"thread_start_us":79,"threads_started":1}
I20260808 16:57:00.880014 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.855648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:59988)
0808 16:57:00.856706 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.856716 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.856803 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.856981 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.856981 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.857165 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:00.857237 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.857239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.858179 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.858180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.859085 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.859088 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.859673 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.859674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.859754 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.860191 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:00.860197 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:00.875577 (+ 15380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:00.877645 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.877646 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.877647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.878024 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.878238 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.878239 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.878239 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.878294 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.878585 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:00.878585 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:00.878682 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.879536 (+ 854us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.879914 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1040,"mutex_wait_us":48,"thread_start_us":83,"threads_started":1}
I20260808 16:57:00.886962 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 17301
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208219, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.126@KRBTEST.COM for oryx/127.8.14.124@KRBTEST.COM
I20260808 16:57:00.906383 18057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.882137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46491 (local address 127.0.0.1:56846)
0808 16:57:00.882194 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.882206 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.882251 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.883847 (+ 1596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.883848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.883972 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:00.884012 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.884014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.884877 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.884877 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.885361 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.885362 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.885813 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.885813 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.885860 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.886229 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:00.886233 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:00.903143 (+ 16910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:00.905265 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.905267 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.905268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.905502 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.905701 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.905702 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.905702 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.905757 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.906035 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:00.906036 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:00.906084 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.906146 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.906261 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 16:57:00.906764 18298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.883653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56846 (local address 127.8.14.124:46491)
0808 16:57:00.883713 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.883715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.883724 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:57:00.883754 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.883755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.883764 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.883829 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.884087 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.884797 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.885438 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.885732 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.885799 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.903310 (+ 17511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.903312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.903312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.903321 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.905168 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.905581 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.905583 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.905583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.905636 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.905821 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.905822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.905822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.905953 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.906010 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.906594 (+ 584us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.906674 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 16:57:00.907256 17209 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.14.124:46491: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "abd97edeb20a4efd922778124c11f19b"
member_type: NON_VOTER
last_known_addr {
host: "127.8.14.124"
port: 46491
}
attrs {
promote: true
}
}
dest_uuid: "1808f7e1cd1845c1a3da149aca6a718e"
cas_config_opid_index: 5
I20260808 16:57:00.907358 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:43969
--local_ip_for_outbound_sockets=127.8.14.65
--tserver_master_addrs=127.8.14.126:42179
--webserver_port=37511
--webserver_interface=127.8.14.65
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.65.keytab
--principal=oryx/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.14.126:42179,127.8.14.125:35539
--tserver_master_addrs=127.8.14.126:42179,127.8.14.125:35539,127.8.14.124:46491 with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:57:00.907516 17209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: NON_VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: true } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } }
I20260808 16:57:00.907920 18060 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER abd97edeb20a4efd922778124c11f19b (127.8.14.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: NON_VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: true } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } } }
I20260808 16:57:00.909086 18060 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.14.124:46491
I20260808 16:57:00.909175 18060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1808f7e1cd1845c1a3da149aca6a718e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: NON_VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: true } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } } }
I20260808 16:57:00.909262 18060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [sys.catalog]: This master's current role is: LEADER
I20260808 16:57:00.909590 18197 master.cc:537] Master@127.8.14.124:46491 shutting down...
I20260808 16:57:00.909837 18256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:00.910817 18256 raft_consensus.cc:1261] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b [term 1 FOLLOWER]: Refusing update from remote peer 1808f7e1cd1845c1a3da149aca6a718e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 16:57:00.911113 18327 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [LEADER]: Connected to new peer: Peer: permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 16:57:00.911760 17192 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e -> Peer abd97edeb20a4efd922778124c11f19b (127.8.14.124:46491): Couldn't send request to peer abd97edeb20a4efd922778124c11f19b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 16:57:00.912964 18057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.909203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:35539 (local address 127.0.0.1:35746)
0808 16:57:00.909298 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.909309 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.909353 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.909676 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.909677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.909679 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:00.909724 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.909725 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.910712 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.910713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.911893 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.911896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.912690 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.912691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.912766 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.912791 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.912858 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.912895 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 16:57:00.913340 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.909444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35746 (local address 127.8.14.125:35539)
0808 16:57:00.909488 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.909490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.909498 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:57:00.909524 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.909525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.909551 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.909614 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:00.909877 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.910597 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.911991 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.912597 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.912675 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.912749 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.913205 (+ 456us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.913255 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:57:00.913362 18197 raft_consensus.cc:2229] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:57:00.913450 18197 raft_consensus.cc:2258] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:00.913502 18197 tablet_replica.cc:330] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: stopping tablet replica
I20260808 16:57:00.913570 18132 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 LEARNER]: Refusing update from remote peer 1808f7e1cd1845c1a3da149aca6a718e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 16:57:00.913800 18327 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: NON_VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 16:57:00.914347 18132 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER abd97edeb20a4efd922778124c11f19b (127.8.14.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: NON_VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: true } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } } }
I20260808 16:57:00.915721 18330 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1808f7e1cd1845c1a3da149aca6a718e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: NON_VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: true } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } } }
I20260808 16:57:00.915890 18330 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [sys.catalog]: This master's current role is: LEARNER
I20260808 16:57:00.916042 18176 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1808f7e1cd1845c1a3da149aca6a718e. Latest consensus state: current_term: 1 leader_uuid: "1808f7e1cd1845c1a3da149aca6a718e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: NON_VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: true } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } } }
I20260808 16:57:00.916167 18176 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [sys.catalog]: This master's current role is: LEARNER
I20260808 16:57:00.916795 18058 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e: attempting to promote NON_VOTER a2cc6282c44644d4b7a4f199a38d6bc8 to VOTER
I20260808 16:57:00.916983 18058 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: false } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } }
I20260808 16:57:00.917418 18057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.917316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46491 (local address 127.0.0.1:56856)
0808 16:57:00.917348 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.917392 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 16:57:00.917673 18132 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 LEARNER]: Refusing update from remote peer 1808f7e1cd1845c1a3da149aca6a718e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 16:57:00.917904 18060 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 16:57:00.917954 18057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.917658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46491 (local address 127.0.0.1:56864)
0808 16:57:00.917902 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.917929 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232}
W20260808 16:57:00.918349 17192 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e -> Peer abd97edeb20a4efd922778124c11f19b (127.8.14.124:46491): Couldn't send request to peer abd97edeb20a4efd922778124c11f19b. Status: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46491: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 16:57:00.918571 18060 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a2cc6282c44644d4b7a4f199a38d6bc8 (127.8.14.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: false } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } } }
I20260808 16:57:00.918716 18132 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a2cc6282c44644d4b7a4f199a38d6bc8 (127.8.14.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: false } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } } }
I20260808 16:57:00.919589 18060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1808f7e1cd1845c1a3da149aca6a718e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: false } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } } }
I20260808 16:57:00.919659 18060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [sys.catalog]: This master's current role is: LEADER
I20260808 16:57:00.919710 18060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1808f7e1cd1845c1a3da149aca6a718e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: false } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } } }
I20260808 16:57:00.919751 18060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e [sys.catalog]: This master's current role is: LEADER
I20260808 16:57:00.920025 18176 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1808f7e1cd1845c1a3da149aca6a718e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: false } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } } }
I20260808 16:57:00.920145 18176 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:57:00.927359 18197 master.cc:552] Master@127.8.14.124:46491 shutdown complete.
I20260808 16:57:00.927412 18197 master_runner.cc:305] Clearing existing system tablet
I20260808 16:57:00.927546 18197 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:00.927591 18197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:00.928339 18197 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:57:00.928882 18333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:00.929072 18197 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:00.929140 18197 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:57:00.929171 18197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/wal
uuid: "abd97edeb20a4efd922778124c11f19b"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "996c248681956f8fc38e316e6126f6e3"
server_key_iv: "58c4543a5484e993d80e7db022818ae5"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.930053 18197 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:00.931384 18197 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 16:57:00.931456 18197 log.cc:1196] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 16:57:00.931654 18197 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: Deleting consensus metadata
I20260808 16:57:00.931767 18197 master_runner.cc:315] Copying system tablet from 127.8.14.126:42179
I20260808 16:57:00.932417 18197 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: tablet copy: Beginning tablet copy session from remote peer at address 127.8.14.126:42179
I20260808 16:57:00.939908 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.932764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60004 (local address 127.8.14.126:42179)
0808 16:57:00.932812 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:00.932815 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:00.932890 (+ 75us) server_negotiation.cc:407] Connection header received
0808 16:57:00.932954 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:00.932955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:00.932965 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:00.933033 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:00.933631 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.934383 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.935430 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.935720 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.935797 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.936885 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:00.936887 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:00.936887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:00.936894 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:00.938599 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.939067 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.939068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.939068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.939146 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:00.939381 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:00.939381 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:00.939382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:00.939507 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:00.939576 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:00.939745 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 16:57:00.939824 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:57:00.939913 18345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:00.932707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:60004)
0808 16:57:00.932830 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:00.932842 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:00.932903 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:00.933037 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:00.933038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:00.933249 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:00.933543 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.933545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.934472 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.934473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.935334 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:00.935337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:00.935802 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:00.935803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:00.935862 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:00.936392 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:00.936398 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:00.936793 (+ 395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:00.938696 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.938697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.938698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.938999 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.939220 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:00.939221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:00.939221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:00.939291 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:00.939580 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:00.939581 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:00.939629 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 16:57:00.939705 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 16:57:00.939807 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
I20260808 16:57:00.940660 17241 tablet_copy_service.cc:140] P 1808f7e1cd1845c1a3da149aca6a718e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer abd97edeb20a4efd922778124c11f19b ({username='oryx', principal='oryx/127.8.14.124@KRBTEST.COM'} at 127.0.0.1:60004)
I20260808 16:57:00.940726 17241 tablet_copy_service.cc:161] P 1808f7e1cd1845c1a3da149aca6a718e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer abd97edeb20a4efd922778124c11f19b at {username='oryx', principal='oryx/127.8.14.124@KRBTEST.COM'} at 127.0.0.1:60004: session id = abd97edeb20a4efd922778124c11f19b-00000000000000000000000000000000
I20260808 16:57:00.941304 17241 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 16:57:00.941789 18197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 16:57:00.943039 18197 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: tablet copy: Starting download of 0 data blocks...
I20260808 16:57:00.943204 18197 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: tablet copy: Starting download of 1 WAL segments...
I20260808 16:57:00.943903 18197 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 16:57:00.944769 17241 tablet_copy_service.cc:342] P 1808f7e1cd1845c1a3da149aca6a718e: Request end of tablet copy session abd97edeb20a4efd922778124c11f19b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.14.124@KRBTEST.COM'} at 127.0.0.1:60004
I20260808 16:57:00.944828 17241 tablet_copy_service.cc:433] P 1808f7e1cd1845c1a3da149aca6a718e: ending tablet copy session abd97edeb20a4efd922778124c11f19b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer abd97edeb20a4efd922778124c11f19b
I20260808 16:57:00.947877 18197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:00.949020 18346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:00.951846 18347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:00.952028 18349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:00.955065 18197 server_base.cc:1034] running on GCE node
I20260808 16:57:00.955168 18197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:00.955313 18197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:00.956449 18197 hybrid_clock.cc:648] HybridClock initialized: now 1786208220956440 us; error 24 us; skew 500 ppm
Aug 08 16:57:00 dist-test-slave-1nl6 krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208220, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:00.958133 18197 init.cc:377] Logged in from keytab as oryx/127.8.14.124@KRBTEST.COM (short username oryx)
I20260808 16:57:00.958415 18197 webserver.cc:460] Webserver started at http://127.8.14.124:41697/ using document root <none> and password file <none>
I20260808 16:57:00.958526 18197 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:00.958556 18197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:00.959434 18197 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:57:00.960052 18359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:00.960220 18197 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:57:00.960276 18197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/wal
uuid: "abd97edeb20a4efd922778124c11f19b"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "996c248681956f8fc38e316e6126f6e3"
server_key_iv: "58c4543a5484e993d80e7db022818ae5"
server_key_version: "encryptionkey@0"
I20260808 16:57:00.960330 18197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:00.984105 18197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:00.986202 18197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:00.989398 18197 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.124:46491
I20260808 16:57:00.989821 18197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/master-2/data/info.pb
I20260808 16:57:00.990929 18451 sys_catalog.cc:263] Verifying existing consensus state
I20260808 16:57:00.991113 18449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.124:46491 every 8 connection(s)
I20260808 16:57:00.991539 18451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: Bootstrap starting.
I20260808 16:57:00.995002 18451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:57:00.995227 18451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: Bootstrap complete.
I20260808 16:57:00.995448 18451 raft_consensus.cc:348] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: false } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } }
I20260808 16:57:00.995544 18451 raft_consensus.cc:729] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: abd97edeb20a4efd922778124c11f19b, State: Initialized, Role: LEARNER
I20260808 16:57:00.995582 18451 consensus_queue.cc:260] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: false } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } }
I20260808 16:57:00.995755 18451 sys_catalog.cc:564] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:57:00.995771 18455 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: false } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } } }
I20260808 16:57:00.996007 18455 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b [sys.catalog]: This master's current role is: LEARNER
I20260808 16:57:00.997304 18469 catalog_manager.cc:1259] Loaded cluster ID: b541f111632a432f91a657c16c22ed0f
I20260808 16:57:00.998018 18469 catalog_manager.cc:1552] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: loading cluster ID for follower catalog manager: success
I20260808 16:57:00.999233 18469 catalog_manager.cc:1574] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: acquiring CA information for follower catalog manager: success
I20260808 16:57:00.999600 18469 catalog_manager.cc:1602] T 00000000000000000000000000000000 P abd97edeb20a4efd922778124c11f19b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 16:57:01.009099 18328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:01.009284 18328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:01.009307 18328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:01.010371 18328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:01.010403 18328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:01.010428 18328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:57:01.011971 18328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.65.keytab
--principal=oryx/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:43969
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=37511
--tserver_master_addrs=127.8.14.126:42179,127.8.14.125:35539,127.8.14.124:46491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:01.012183 18328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:01.012415 18328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:01.017220 18474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:01.017369 18477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:01.017278 18475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:01.017678 18328 server_base.cc:1034] running on GCE node
I20260808 16:57:01.017900 18328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:01.018148 18328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:01.019325 18328 hybrid_clock.cc:648] HybridClock initialized: now 1786208221019302 us; error 38 us; skew 500 ppm
Aug 08 16:57:01 dist-test-slave-1nl6 krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208221, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:01.022193 18328 init.cc:377] Logged in from keytab as oryx/127.8.14.65@KRBTEST.COM (short username oryx)
I20260808 16:57:01.022615 18328 webserver.cc:460] Webserver started at http://127.8.14.65:37511/ using document root <none> and password file <none>
I20260808 16:57:01.022859 18328 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:01.022979 18328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:01.024381 18328 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:57:01.025007 18488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:01.025172 18328 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:57:01.025278 18328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "c196200dfad6458dbbe33229e9f06d7e"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "dbeb0fa300fb90748df76bdafcf91096"
server_key_iv: "f0e533cd10b126ace3303bcb99b2599c"
server_key_version: "encryptionkey@0"
I20260808 16:57:01.025624 18328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:01.041612 18328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:01.043510 18328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:01.043673 18328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:01.043903 18328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:01.044215 18328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:01.044270 18328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:01.044319 18328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:01.044348 18328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:01.050316 18328 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:43969
I20260808 16:57:01.050387 18661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:43969 every 8 connection(s)
I20260808 16:57:01.050722 18328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
Aug 08 16:57:01 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208221, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.125@KRBTEST.COM
Aug 08 16:57:01 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208221, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM
Aug 08 16:57:01 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208221, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.124@KRBTEST.COM
I20260808 16:57:01.058302 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 18328
I20260808 16:57:01.058717 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 17516
I20260808 16:57:01.061582 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.051644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:47189 (local address 127.8.14.126:42179)
0808 16:57:01.051687 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.051689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.051997 (+ 308us) server_negotiation.cc:407] Connection header received
0808 16:57:01.052481 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.052483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.052497 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.052559 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:01.053353 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.054089 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.055181 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.055483 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.055522 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.058111 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:01.058113 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:01.058113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:01.058124 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:01.060377 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.060729 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.060730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.060730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.060773 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.060932 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.060932 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.060933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.061045 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:01.061079 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.061385 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.061459 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:57:01.061884 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.051903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:43583 (local address 127.8.14.125:35539)
0808 16:57:01.051949 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.051951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.052018 (+ 67us) server_negotiation.cc:407] Connection header received
0808 16:57:01.052623 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.052624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.052636 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.052679 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:01.053325 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.053817 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.054790 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.055083 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.055160 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.057552 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:01.057554 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:01.057554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:01.057560 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:01.059754 (+ 2194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.060470 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.060471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.060472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.060543 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.061018 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.061019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.061020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.061226 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:01.061280 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.061585 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.061745 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 16:57:01.062186 18675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.051524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:47189)
0808 16:57:01.051928 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:01.051938 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:01.052389 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:01.052638 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:01.052642 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:01.053111 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:01.053226 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.053232 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.054178 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.054179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.055082 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.055085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.055601 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.055602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.055667 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.056068 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:01.056072 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:01.057943 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:01.060459 (+ 2516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.060460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.060460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.060651 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.060831 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.060832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.060832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.060880 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.061114 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:01.061120 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:01.061256 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 16:57:01.061709 (+ 453us) client_negotiation.cc:240] Negotiation successful
0808 16:57:01.061886 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":7,"thread_start_us":96,"threads_started":1}
I20260808 16:57:01.062430 18673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.051822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:35539 (local address 127.8.14.65:43583)
0808 16:57:01.051943 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:01.051953 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:01.052393 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:01.052719 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:01.052720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:01.053037 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:01.053225 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.053229 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.053922 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.053923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.054678 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.054680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.055167 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.055168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.055298 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.055841 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:01.055851 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:01.057465 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:01.059910 (+ 2445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.059918 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.059921 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.060362 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.060679 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.060680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.060680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.060858 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.061392 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:01.061393 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:01.061448 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 16:57:01.061990 (+ 542us) client_negotiation.cc:240] Negotiation successful
0808 16:57:01.062198 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260808 16:57:01.064971 18671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.051514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:43791 (local address 127.8.14.124:46491)
0808 16:57:01.051652 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.051654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.052513 (+ 859us) server_negotiation.cc:407] Connection header received
0808 16:57:01.052540 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.052542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.052555 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.052620 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:01.053354 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.054333 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.055289 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.055590 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.055646 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.058633 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:01.058634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:01.058634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:01.058643 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:01.063831 (+ 5188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.064166 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.064167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.064167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.064221 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.064446 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.064447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.064447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.064560 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:01.064630 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.064795 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.064871 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260808 16:57:01.064982 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.051524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46491 (local address 127.8.14.65:43791)
0808 16:57:01.051763 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:01.051814 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:01.052371 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:01.052637 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:01.052642 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:01.053076 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:01.053226 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.053232 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.054415 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.054416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.055212 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.055214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.055697 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.055698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.055784 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.056264 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:01.056268 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:01.058555 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:01.063924 (+ 5369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.063925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.063926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.064088 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.064315 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.064315 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.064316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.064375 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.064634 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:01.064635 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:01.064675 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 16:57:01.064756 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 16:57:01.064861 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":21,"thread_start_us":85,"threads_started":1}
I20260808 16:57:01.065704 18663 heartbeater.cc:347] Connected to a master server at 127.8.14.124:46491
I20260808 16:57:01.065809 18663 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:01.065999 18663 heartbeater.cc:511] Master 127.8.14.124:46491 requested a full tablet report, sending...
I20260808 16:57:01.066488 18392 ts_manager.cc:194] Registered new tserver with Master: c196200dfad6458dbbe33229e9f06d7e (127.8.14.65:43969)
I20260808 16:57:01.078455 18664 heartbeater.cc:347] Connected to a master server at 127.8.14.126:42179
I20260808 16:57:01.078451 18668 heartbeater.cc:347] Connected to a master server at 127.8.14.125:35539
I20260808 16:57:01.078540 18664 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:01.078564 18668 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:01.078621 18664 heartbeater.cc:511] Master 127.8.14.126:42179 requested a full tablet report, sending...
I20260808 16:57:01.078642 18668 heartbeater.cc:511] Master 127.8.14.125:35539 requested a full tablet report, sending...
I20260808 16:57:01.078990 17209 ts_manager.cc:194] Re-registered known tserver with Master: c196200dfad6458dbbe33229e9f06d7e (127.8.14.65:43969)
I20260808 16:57:01.079062 18117 ts_manager.cc:194] Registered new tserver with Master: c196200dfad6458dbbe33229e9f06d7e (127.8.14.65:43969)
I20260808 16:57:01.079465 17209 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:47189
I20260808 16:57:01.082427 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:35569
--local_ip_for_outbound_sockets=127.8.14.66
--tserver_master_addrs=127.8.14.126:42179
--webserver_port=40695
--webserver_interface=127.8.14.66
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.66.keytab
--principal=oryx/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.14.126:42179,127.8.14.125:35539
--tserver_master_addrs=127.8.14.126:42179,127.8.14.125:35539,127.8.14.124:46491 with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:01.167682 18677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:01.167876 18677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:01.167899 18677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:01.169152 18677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:01.169195 18677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:01.169242 18677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:57:01.170703 18677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.66.keytab
--principal=oryx/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:35569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=40695
--tserver_master_addrs=127.8.14.126:42179,127.8.14.125:35539,127.8.14.124:46491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:01.170989 18677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:01.171234 18677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:01.176178 18689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:01.176229 18686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:01.176358 18677 server_base.cc:1034] running on GCE node
W20260808 16:57:01.176404 18687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:01.176626 18677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:01.176833 18677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:01.177973 18677 hybrid_clock.cc:648] HybridClock initialized: now 1786208221177959 us; error 28 us; skew 500 ppm
Aug 08 16:57:01 dist-test-slave-1nl6 krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208221, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:01.180388 18677 init.cc:377] Logged in from keytab as oryx/127.8.14.66@KRBTEST.COM (short username oryx)
I20260808 16:57:01.180743 18677 webserver.cc:460] Webserver started at http://127.8.14.66:40695/ using document root <none> and password file <none>
I20260808 16:57:01.180917 18677 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:01.180966 18677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:01.182184 18677 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:57:01.182730 18697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:01.182940 18677 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:57:01.183048 18677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "cb7e31d9946142de9f8efa0adc06ab78"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "f7e49fb46116dd696260c09a003c8072"
server_key_iv: "444ed3c23250ef16c63780961c670057"
server_key_version: "encryptionkey@0"
I20260808 16:57:01.183380 18677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:01.214840 18677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:01.217279 18677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:01.217445 18677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:01.217698 18677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:01.218020 18677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:01.218079 18677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:01.218122 18677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:01.218148 18677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:01.224128 18677 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:35569
I20260808 16:57:01.224171 18873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:35569 every 8 connection(s)
I20260808 16:57:01.224548 18677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:57:01.228008 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 18677
I20260808 16:57:01.228168 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 17709
Aug 08 16:57:01 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208221, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM
Aug 08 16:57:01 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208221, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.125@KRBTEST.COM
I20260808 16:57:01.237046 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.225352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:38389 (local address 127.8.14.126:42179)
0808 16:57:01.225403 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.225405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.225619 (+ 214us) server_negotiation.cc:407] Connection header received
0808 16:57:01.226730 (+ 1111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.226731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.226745 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.226777 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:01.227633 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.228408 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.229715 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.229927 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.230005 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.232580 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:01.232582 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:01.232583 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:01.232591 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:01.234667 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.235259 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.235260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.235260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.235315 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.235940 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.235941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.235942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.236400 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:01.236468 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.236846 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.236927 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 16:57:01.237697 18884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.225292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:38389)
0808 16:57:01.225474 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:01.225510 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:01.226601 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:01.226869 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:01.226869 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:01.227317 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:01.227521 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.227527 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.228537 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.228538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.229631 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.229634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.230011 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.230012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.230144 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.230676 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:01.230685 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:01.232462 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:01.234772 (+ 2310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.234803 (+ 31us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.234805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.235173 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.235395 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.235396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.235397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.235846 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.236507 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:01.236514 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:01.236669 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 16:57:01.237197 (+ 528us) client_negotiation.cc:240] Negotiation successful
0808 16:57:01.237392 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":108,"spinlock_wait_cycles":6912,"thread_start_us":102,"threads_started":1}
I20260808 16:57:01.238467 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.225768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:53831 (local address 127.8.14.125:35539)
0808 16:57:01.225815 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.225817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.225826 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:57:01.226730 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.226731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.226745 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.226777 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:01.227632 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.228416 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.232584 (+ 4168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.232888 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.232932 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.235194 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:01.235195 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:01.235196 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:01.235203 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:01.236956 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.237485 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.237486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.237486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.237543 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.237732 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.237733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.237733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.237862 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:01.237914 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.238265 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.238359 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:57:01.238967 18887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.225592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:35539 (local address 127.8.14.66:53831)
0808 16:57:01.225739 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:01.225749 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:01.226603 (+ 854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:01.226861 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:01.226866 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:01.227326 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:01.227521 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.227527 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.228561 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.228561 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.232494 (+ 3933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.232497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.233022 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.233023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.233094 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.233516 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:01.233521 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:01.235092 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:01.237033 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.237034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.237034 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.237394 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.237608 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.237608 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.237609 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.237660 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.238098 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:01.238099 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:01.238143 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 16:57:01.238690 (+ 547us) client_negotiation.cc:240] Negotiation successful
0808 16:57:01.238808 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":2960,"thread_start_us":75,"threads_started":1}
I20260808 16:57:01.239480 18875 heartbeater.cc:347] Connected to a master server at 127.8.14.126:42179
I20260808 16:57:01.239599 18875 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:01.239776 18875 heartbeater.cc:511] Master 127.8.14.126:42179 requested a full tablet report, sending...
I20260808 16:57:01.240168 17209 ts_manager.cc:194] Re-registered known tserver with Master: cb7e31d9946142de9f8efa0adc06ab78 (127.8.14.66:35569)
I20260808 16:57:01.240612 17209 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:38389
I20260808 16:57:01.242741 18878 heartbeater.cc:347] Connected to a master server at 127.8.14.125:35539
I20260808 16:57:01.242797 18878 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:01.242869 18878 heartbeater.cc:511] Master 127.8.14.125:35539 requested a full tablet report, sending...
I20260808 16:57:01.243192 18117 ts_manager.cc:194] Registered new tserver with Master: cb7e31d9946142de9f8efa0adc06ab78 (127.8.14.66:35569)
Aug 08 16:57:01 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208221, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.124@KRBTEST.COM
I20260808 16:57:01.248039 18671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.225314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:37301 (local address 127.8.14.124:46491)
0808 16:57:01.225363 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.225365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.225701 (+ 336us) server_negotiation.cc:407] Connection header received
0808 16:57:01.226716 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.226717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.226732 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.226777 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:01.227631 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.228368 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.242671 (+ 14303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.243119 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.243213 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.245123 (+ 1910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:01.245124 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:01.245124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:01.245132 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:01.246737 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.247188 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.247189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.247190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.247262 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.247475 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.247476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.247476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.247628 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:01.247696 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.247800 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.247900 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:57:01.248152 18885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.225292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46491 (local address 127.8.14.66:37301)
0808 16:57:01.225628 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:01.225640 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:01.226608 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:01.226866 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:01.226867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:01.227274 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:01.227520 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.227525 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.228498 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.228499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.231688 (+ 3189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.231691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.243235 (+ 11544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.243236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.243329 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.243763 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:01.243768 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:01.245014 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:01.246832 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.246833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.246834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.247106 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.247346 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.247346 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.247347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.247399 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.247688 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:01.247689 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:01.247720 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 16:57:01.247765 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 16:57:01.247857 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":2223,"thread_start_us":95,"threads_started":1}
I20260808 16:57:01.248507 18874 heartbeater.cc:347] Connected to a master server at 127.8.14.124:46491
I20260808 16:57:01.248565 18874 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:01.248617 18874 heartbeater.cc:511] Master 127.8.14.124:46491 requested a full tablet report, sending...
I20260808 16:57:01.248924 18392 ts_manager.cc:194] Registered new tserver with Master: cb7e31d9946142de9f8efa0adc06ab78 (127.8.14.66:35569)
I20260808 16:57:01.251451 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:33471
--local_ip_for_outbound_sockets=127.8.14.67
--tserver_master_addrs=127.8.14.126:42179
--webserver_port=34775
--webserver_interface=127.8.14.67
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.67.keytab
--principal=oryx/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.14.126:42179,127.8.14.125:35539
--tserver_master_addrs=127.8.14.126:42179,127.8.14.125:35539,127.8.14.124:46491 with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:01.333920 18888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:01.334142 18888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:01.334183 18888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:01.335397 18888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:01.335457 18888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:01.335505 18888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:57:01.337196 18888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.67.keytab
--principal=oryx/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:33471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=34775
--tserver_master_addrs=127.8.14.126:42179,127.8.14.125:35539,127.8.14.124:46491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:01.337524 18888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:01.337766 18888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:01.343341 18897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:01.343386 18901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:01.343417 18888 server_base.cc:1034] running on GCE node
W20260808 16:57:01.343355 18898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:01.343719 18888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:01.343957 18888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:01.345113 18888 hybrid_clock.cc:648] HybridClock initialized: now 1786208221345087 us; error 32 us; skew 500 ppm
Aug 08 16:57:01 dist-test-slave-1nl6 krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208221, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:01.347730 18888 init.cc:377] Logged in from keytab as oryx/127.8.14.67@KRBTEST.COM (short username oryx)
I20260808 16:57:01.348156 18888 webserver.cc:460] Webserver started at http://127.8.14.67:34775/ using document root <none> and password file <none>
I20260808 16:57:01.348405 18888 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:01.348480 18888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:01.349750 18888 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:57:01.350342 18912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:01.350518 18888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:57:01.350631 18888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "ae940e53ec88472ba06e15f8e4b95748"
format_stamp: "Formatted at 2026-08-08 16:57:00 on dist-test-slave-1nl6"
server_key: "e03eabb1529605a9a50a61a0eaf918ea"
server_key_iv: "9a0f0cdfa91d9795586f5947ee54171a"
server_key_version: "encryptionkey@0"
I20260808 16:57:01.351012 18888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:01.357234 18888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:01.358875 18888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:01.359086 18888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:01.359297 18888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:01.359602 18888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:01.359668 18888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:01.359707 18888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:01.359733 18888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:01.365873 18888 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:33471
I20260808 16:57:01.365907 19088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:33471 every 8 connection(s)
I20260808 16:57:01.366250 18888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
Aug 08 16:57:01 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208221, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.125@KRBTEST.COM
Aug 08 16:57:01 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208221, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM
Aug 08 16:57:01 dist-test-slave-1nl6 krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208221, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.124@KRBTEST.COM
I20260808 16:57:01.376644 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 18888
I20260808 16:57:01.377728 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.367016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:56619 (local address 127.8.14.125:35539)
0808 16:57:01.367065 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.367068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.367385 (+ 317us) server_negotiation.cc:407] Connection header received
0808 16:57:01.368562 (+ 1177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.368563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.368577 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.368643 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:01.369435 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.369929 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.371412 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.371722 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.371800 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.374273 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:01.374274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:01.374274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:01.374280 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:01.376049 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.376556 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.376558 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.376558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.376621 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.377009 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.377010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.377011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.377146 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:01.377193 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.377594 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.377666 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:57:01.378257 18671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.366987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:44617 (local address 127.8.14.124:46491)
0808 16:57:01.367046 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.367049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.367342 (+ 293us) server_negotiation.cc:407] Connection header received
0808 16:57:01.368585 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.368587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.368600 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.368636 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:01.369443 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.370204 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.371862 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.372130 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.372228 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.375257 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:01.375259 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:01.375259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:01.375267 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:01.376954 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.377392 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.377394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.377394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.377453 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.377658 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.377659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.377659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.377794 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:01.377849 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.378055 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.378140 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:57:01.378405 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.367427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:48559 (local address 127.8.14.126:42179)
0808 16:57:01.367472 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.367475 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.367585 (+ 110us) server_negotiation.cc:407] Connection header received
0808 16:57:01.368585 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.368587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.368600 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.368643 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:01.369442 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.370204 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.371956 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.372272 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.372334 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.375476 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:01.375478 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:01.375478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:01.375486 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:01.377136 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.377518 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.377519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.377520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.377572 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.377786 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.377787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.377788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.377948 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:01.378014 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.378227 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.378318 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:57:01.378533 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.367364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:48559)
0808 16:57:01.367523 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:01.367533 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:01.368460 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:01.368762 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:01.368767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:01.369212 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:01.369333 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.369339 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.371117 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.371118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.371868 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.371871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.372354 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.372356 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.372424 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.372870 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:01.372874 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:01.375384 (+ 2510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:01.377218 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.377219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.377220 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.377428 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.377648 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.377649 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.377649 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.377702 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.378043 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:01.378044 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:01.378087 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 16:57:01.378149 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 16:57:01.378311 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":111,"thread_start_us":69,"threads_started":1}
I20260808 16:57:01.378670 19098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.366948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46491 (local address 127.8.14.67:44617)
0808 16:57:01.367238 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:01.367254 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:01.368460 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:01.368762 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:01.368767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:01.369216 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:01.369333 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.369339 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.370296 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.370297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.371303 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.371306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.372531 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.372532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.372619 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.373056 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:01.373061 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:01.375173 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:01.377055 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.377056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.377057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.377305 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.377525 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.377525 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.377526 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.377580 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.377885 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:01.377886 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:01.377931 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 16:57:01.378411 (+ 480us) client_negotiation.cc:240] Negotiation successful
0808 16:57:01.378525 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":31,"thread_start_us":94,"threads_started":1}
I20260808 16:57:01.379104 19091 heartbeater.cc:347] Connected to a master server at 127.8.14.126:42179
I20260808 16:57:01.379201 19091 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:01.379403 19091 heartbeater.cc:511] Master 127.8.14.126:42179 requested a full tablet report, sending...
I20260808 16:57:01.379783 17209 ts_manager.cc:194] Re-registered known tserver with Master: ae940e53ec88472ba06e15f8e4b95748 (127.8.14.67:33471)
I20260808 16:57:01.380309 17209 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:48559
I20260808 16:57:01.380856 19097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.366941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:35539 (local address 127.8.14.67:56619)
0808 16:57:01.367212 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:01.367254 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:01.368459 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:01.368762 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:01.368768 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:01.369150 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:01.369332 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.369336 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.370037 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.370038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.371303 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.371306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.371813 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.371814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.371937 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.372546 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:01.372554 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:01.374173 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:01.376141 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.376148 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.376150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.376455 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.376702 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:01.376703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:01.376755 (+ 52us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:01.376920 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:01.377250 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:01.377257 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:01.377425 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 16:57:01.378078 (+ 653us) client_negotiation.cc:240] Negotiation successful
0808 16:57:01.380722 (+ 2644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":9,"thread_start_us":91,"threads_started":1}
I20260808 16:57:01.381083 19089 heartbeater.cc:347] Connected to a master server at 127.8.14.124:46491
I20260808 16:57:01.381402 19094 heartbeater.cc:347] Connected to a master server at 127.8.14.125:35539
I20260808 16:57:01.381484 19089 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:01.381520 19094 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:01.381569 19094 heartbeater.cc:511] Master 127.8.14.125:35539 requested a full tablet report, sending...
I20260808 16:57:01.381562 19089 heartbeater.cc:511] Master 127.8.14.124:46491 requested a full tablet report, sending...
I20260808 16:57:01.381907 18117 ts_manager.cc:194] Registered new tserver with Master: ae940e53ec88472ba06e15f8e4b95748 (127.8.14.67:33471)
I20260808 16:57:01.381987 18392 ts_manager.cc:194] Registered new tserver with Master: ae940e53ec88472ba06e15f8e4b95748 (127.8.14.67:33471)
I20260808 16:57:01.386963 18671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.377997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56880 (local address 127.8.14.124:46491)
0808 16:57:01.378353 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.378355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.379334 (+ 979us) server_negotiation.cc:407] Connection header received
0808 16:57:01.379395 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.379396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.379406 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.379450 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:01.380126 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.380826 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.381865 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.382136 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.382178 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.383589 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:01.383590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:01.383590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:01.383597 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:01.385479 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.385956 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.385957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.385958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.386020 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.386262 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.386263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.386263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.386419 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:01.386495 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.386734 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.386820 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":67,"threads_started":1}
I20260808 16:57:01.388164 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.378424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60008 (local address 127.8.14.126:42179)
0808 16:57:01.378479 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.378481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.379475 (+ 994us) server_negotiation.cc:407] Connection header received
0808 16:57:01.379566 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.379567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.379577 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.379605 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:01.380455 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.381166 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.382867 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.383117 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.383204 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.384572 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:01.384573 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:01.384574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:01.384581 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:01.386192 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.386532 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.386533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.386533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.386592 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.387662 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.387663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.387664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.387780 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:01.387886 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.387992 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.388068 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41,"thread_start_us":99,"threads_started":1}
I20260808 16:57:01.389420 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.379085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35762 (local address 127.8.14.125:35539)
0808 16:57:01.380581 (+ 1496us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.380583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.380592 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:57:01.380611 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.380612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.380621 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.380674 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:01.382959 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.383710 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.384693 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.384958 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.385026 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.386438 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:01.386439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:01.386440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:01.386447 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:01.388037 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.388469 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.388470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.388470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.388525 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.388760 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.388761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.388761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.388890 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:01.388962 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.389204 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.389304 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1475}
I20260808 16:57:01.391633 17209 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:57:01.393321 17209 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:57:01.400560 18057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.397105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:35569 (local address 127.0.0.1:52430)
0808 16:57:01.397136 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:01.397146 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:01.397186 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:01.397591 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:01.397592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:01.397593 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:01.397630 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.397632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.398715 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.398716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.399639 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.399642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.400316 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.400317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.400378 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.400383 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:57:01.400451 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 16:57:01.400494 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260808 16:57:01.400576 19123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.397213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52430 (local address 127.8.14.66:35569)
0808 16:57:01.397394 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.397399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.397426 (+ 27us) server_negotiation.cc:407] Connection header received
0808 16:57:01.397467 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.397469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.397484 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.397595 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:01.397734 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.398637 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.399705 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.400200 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.400314 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.400398 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.400465 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.400502 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":111,"threads_started":1}
I20260808 16:57:01.401408 18778 tablet_service.cc:1467] Processing CreateTablet for tablet 809ae4c4e89c4f9eaa5515c31db8ef7d (DEFAULT_TABLE table=test-table [id=6305d9efedcc47e781c56614d5815e1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:01.401710 19131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.397715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56206 (local address 127.8.14.67:33471)
0808 16:57:01.397879 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.397883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.397922 (+ 39us) server_negotiation.cc:407] Connection header received
0808 16:57:01.398004 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.398006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.398020 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.398093 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:01.398238 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.399338 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.400700 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.401392 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.401485 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.401563 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.401599 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.401638 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260808 16:57:01.401793 19130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.397538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:33471 (local address 127.0.0.1:56206)
0808 16:57:01.397875 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:01.397883 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:01.397926 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:01.398109 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:01.398110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:01.398111 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:01.398156 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.398158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.399448 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.399449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.400609 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.400612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.401499 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.401500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.401545 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.401549 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:57:01.401586 (+ 37us) client_negotiation.cc:240] Negotiation successful
0808 16:57:01.401642 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":24,"thread_start_us":73,"threads_started":1}
I20260808 16:57:01.401830 18778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 809ae4c4e89c4f9eaa5515c31db8ef7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:01.403630 18983 tablet_service.cc:1467] Processing CreateTablet for tablet 809ae4c4e89c4f9eaa5515c31db8ef7d (DEFAULT_TABLE table=test-table [id=6305d9efedcc47e781c56614d5815e1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:01.403980 18983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 809ae4c4e89c4f9eaa5515c31db8ef7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:01.404671 19129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.397359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:43969 (local address 127.0.0.1:58526)
0808 16:57:01.397688 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:01.397698 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:01.397742 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:01.398209 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:01.398209 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:01.398212 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:01.398253 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.398255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.399459 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.399460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.400610 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.400612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.404227 (+ 3615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.404228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.404312 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.404317 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:57:01.404387 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 16:57:01.404430 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":2,"thread_start_us":69,"threads_started":1}
I20260808 16:57:01.404979 19127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.397323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58526 (local address 127.8.14.65:43969)
0808 16:57:01.398039 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.398043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.398050 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:57:01.398110 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.398112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.398127 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.398211 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:01.398324 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.399375 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.400880 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.403747 (+ 2867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.403820 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.404183 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.404424 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.404482 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":94,"threads_started":1}
I20260808 16:57:01.405206 19134 tablet_bootstrap.cc:492] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78: Bootstrap starting.
I20260808 16:57:01.405651 18564 tablet_service.cc:1467] Processing CreateTablet for tablet 809ae4c4e89c4f9eaa5515c31db8ef7d (DEFAULT_TABLE table=test-table [id=6305d9efedcc47e781c56614d5815e1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:01.405764 19134 tablet_bootstrap.cc:654] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:01.405962 18564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 809ae4c4e89c4f9eaa5515c31db8ef7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:01.405984 19134 log.cc:824] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78: Log is configured to *not* fsync() on all Append() calls
I20260808 16:57:01.406646 19134 tablet_bootstrap.cc:492] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78: No bootstrap required, opened a new log
I20260808 16:57:01.406688 19134 ts_tablet_manager.cc:1397] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:57:01.407656 19136 tablet_bootstrap.cc:492] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748: Bootstrap starting.
I20260808 16:57:01.408466 19136 tablet_bootstrap.cc:654] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:01.408771 19136 log.cc:824] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748: Log is configured to *not* fsync() on all Append() calls
I20260808 16:57:01.409070 19134 raft_consensus.cc:348] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb7e31d9946142de9f8efa0adc06ab78" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35569 } } peers { permanent_uuid: "c196200dfad6458dbbe33229e9f06d7e" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43969 } } peers { permanent_uuid: "ae940e53ec88472ba06e15f8e4b95748" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 33471 } }
I20260808 16:57:01.409237 19134 raft_consensus.cc:374] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:01.409281 19134 raft_consensus.cc:729] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb7e31d9946142de9f8efa0adc06ab78, State: Initialized, Role: FOLLOWER
I20260808 16:57:01.409420 19134 consensus_queue.cc:260] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb7e31d9946142de9f8efa0adc06ab78" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35569 } } peers { permanent_uuid: "c196200dfad6458dbbe33229e9f06d7e" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43969 } } peers { permanent_uuid: "ae940e53ec88472ba06e15f8e4b95748" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 33471 } }
I20260808 16:57:01.409554 19136 tablet_bootstrap.cc:492] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748: No bootstrap required, opened a new log
I20260808 16:57:01.409624 19136 ts_tablet_manager.cc:1397] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:57:01.409679 19134 ts_tablet_manager.cc:1428] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:57:01.409719 18875 heartbeater.cc:503] Master 127.8.14.126:42179 was elected leader, sending a full tablet report...
I20260808 16:57:01.410964 19140 tablet_bootstrap.cc:492] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e: Bootstrap starting.
I20260808 16:57:01.411621 19140 tablet_bootstrap.cc:654] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:01.411959 19140 log.cc:824] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e: Log is configured to *not* fsync() on all Append() calls
I20260808 16:57:01.412822 19140 tablet_bootstrap.cc:492] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e: No bootstrap required, opened a new log
I20260808 16:57:01.412876 19140 ts_tablet_manager.cc:1397] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 16:57:01.413040 19136 raft_consensus.cc:348] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb7e31d9946142de9f8efa0adc06ab78" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35569 } } peers { permanent_uuid: "c196200dfad6458dbbe33229e9f06d7e" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43969 } } peers { permanent_uuid: "ae940e53ec88472ba06e15f8e4b95748" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 33471 } }
I20260808 16:57:01.413182 19136 raft_consensus.cc:374] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:01.413228 19136 raft_consensus.cc:729] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae940e53ec88472ba06e15f8e4b95748, State: Initialized, Role: FOLLOWER
I20260808 16:57:01.413338 19136 consensus_queue.cc:260] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb7e31d9946142de9f8efa0adc06ab78" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35569 } } peers { permanent_uuid: "c196200dfad6458dbbe33229e9f06d7e" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43969 } } peers { permanent_uuid: "ae940e53ec88472ba06e15f8e4b95748" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 33471 } }
I20260808 16:57:01.413611 19136 ts_tablet_manager.cc:1428] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 16:57:01.413638 19091 heartbeater.cc:503] Master 127.8.14.126:42179 was elected leader, sending a full tablet report...
I20260808 16:57:01.414779 19140 raft_consensus.cc:348] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb7e31d9946142de9f8efa0adc06ab78" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35569 } } peers { permanent_uuid: "c196200dfad6458dbbe33229e9f06d7e" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43969 } } peers { permanent_uuid: "ae940e53ec88472ba06e15f8e4b95748" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 33471 } }
I20260808 16:57:01.414959 19140 raft_consensus.cc:374] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:01.415001 19140 raft_consensus.cc:729] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c196200dfad6458dbbe33229e9f06d7e, State: Initialized, Role: FOLLOWER
I20260808 16:57:01.415108 19140 consensus_queue.cc:260] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb7e31d9946142de9f8efa0adc06ab78" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35569 } } peers { permanent_uuid: "c196200dfad6458dbbe33229e9f06d7e" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43969 } } peers { permanent_uuid: "ae940e53ec88472ba06e15f8e4b95748" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 33471 } }
I20260808 16:57:01.415369 19140 ts_tablet_manager.cc:1428] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 16:57:01.415438 18664 heartbeater.cc:503] Master 127.8.14.126:42179 was elected leader, sending a full tablet report...
I20260808 16:57:01.433970 19143 raft_consensus.cc:482] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:57:01.434063 19143 raft_consensus.cc:504] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb7e31d9946142de9f8efa0adc06ab78" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35569 } } peers { permanent_uuid: "c196200dfad6458dbbe33229e9f06d7e" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43969 } } peers { permanent_uuid: "ae940e53ec88472ba06e15f8e4b95748" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 33471 } }
I20260808 16:57:01.434372 19143 leader_election.cc:288] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb7e31d9946142de9f8efa0adc06ab78 (127.8.14.66:35569), c196200dfad6458dbbe33229e9f06d7e (127.8.14.65:43969)
I20260808 16:57:01.437013 19098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.434554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:43969 (local address 127.8.14.67:37645)
0808 16:57:01.434600 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:01.434611 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:01.434657 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:01.434842 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:01.434843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:01.434844 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:01.434939 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.434940 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.435537 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.435539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.436265 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.436267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.436796 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.436797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.436878 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.436881 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:57:01.436934 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 16:57:01.436963 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":3}
I20260808 16:57:01.437068 19127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.434645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:37645 (local address 127.8.14.65:43969)
0808 16:57:01.434690 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.434692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.434707 (+ 15us) server_negotiation.cc:407] Connection header received
0808 16:57:01.434726 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.434727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.434752 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.434817 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:01.435024 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.435460 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.436364 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.436718 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.436772 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.436786 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.436957 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.436991 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 16:57:01.437179 19097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.434499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:35569 (local address 127.8.14.67:57745)
0808 16:57:01.434558 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:01.434570 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:01.434615 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:01.434775 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:01.434777 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:01.434778 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:01.434869 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.434871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.435578 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.435579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.436275 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:01.436277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.436888 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.436889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:01.436981 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.436984 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:57:01.437042 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 16:57:01.437086 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":1}
I20260808 16:57:01.437196 19123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.434609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:57745 (local address 127.8.14.66:35569)
0808 16:57:01.434651 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.434653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.434662 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:57:01.434689 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.434690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.434702 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.434762 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:01.434971 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.435473 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.436409 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.436796 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.436871 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.436881 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.437066 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.437110 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:57:01.437531 18808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "809ae4c4e89c4f9eaa5515c31db8ef7d" candidate_uuid: "ae940e53ec88472ba06e15f8e4b95748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb7e31d9946142de9f8efa0adc06ab78" is_pre_election: true
I20260808 16:57:01.437674 18588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "809ae4c4e89c4f9eaa5515c31db8ef7d" candidate_uuid: "ae940e53ec88472ba06e15f8e4b95748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c196200dfad6458dbbe33229e9f06d7e" is_pre_election: true
I20260808 16:57:01.437745 18808 raft_consensus.cc:2454] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae940e53ec88472ba06e15f8e4b95748 in term 0.
I20260808 16:57:01.437831 18588 raft_consensus.cc:2454] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae940e53ec88472ba06e15f8e4b95748 in term 0.
I20260808 16:57:01.437991 18916 leader_election.cc:302] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae940e53ec88472ba06e15f8e4b95748, cb7e31d9946142de9f8efa0adc06ab78; no voters:
I20260808 16:57:01.438112 19143 raft_consensus.cc:2781] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:57:01.438163 19143 raft_consensus.cc:482] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:57:01.438211 19143 raft_consensus.cc:3037] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:01.438934 19143 raft_consensus.cc:504] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb7e31d9946142de9f8efa0adc06ab78" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35569 } } peers { permanent_uuid: "c196200dfad6458dbbe33229e9f06d7e" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43969 } } peers { permanent_uuid: "ae940e53ec88472ba06e15f8e4b95748" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 33471 } }
I20260808 16:57:01.439088 19143 leader_election.cc:288] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [CANDIDATE]: Term 1 election: Requested vote from peers cb7e31d9946142de9f8efa0adc06ab78 (127.8.14.66:35569), c196200dfad6458dbbe33229e9f06d7e (127.8.14.65:43969)
I20260808 16:57:01.439234 18588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "809ae4c4e89c4f9eaa5515c31db8ef7d" candidate_uuid: "ae940e53ec88472ba06e15f8e4b95748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c196200dfad6458dbbe33229e9f06d7e"
I20260808 16:57:01.439263 18808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "809ae4c4e89c4f9eaa5515c31db8ef7d" candidate_uuid: "ae940e53ec88472ba06e15f8e4b95748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb7e31d9946142de9f8efa0adc06ab78"
I20260808 16:57:01.439327 18588 raft_consensus.cc:3037] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:01.439342 18808 raft_consensus.cc:3037] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:01.439981 18808 raft_consensus.cc:2454] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae940e53ec88472ba06e15f8e4b95748 in term 1.
I20260808 16:57:01.439982 18588 raft_consensus.cc:2454] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae940e53ec88472ba06e15f8e4b95748 in term 1.
I20260808 16:57:01.440146 18916 leader_election.cc:302] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae940e53ec88472ba06e15f8e4b95748, cb7e31d9946142de9f8efa0adc06ab78; no voters:
I20260808 16:57:01.440250 19143 raft_consensus.cc:2781] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:57:01.440392 19143 raft_consensus.cc:686] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [term 1 LEADER]: Becoming Leader. State: Replica: ae940e53ec88472ba06e15f8e4b95748, State: Running, Role: LEADER
I20260808 16:57:01.440493 19143 consensus_queue.cc:237] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb7e31d9946142de9f8efa0adc06ab78" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35569 } } peers { permanent_uuid: "c196200dfad6458dbbe33229e9f06d7e" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43969 } } peers { permanent_uuid: "ae940e53ec88472ba06e15f8e4b95748" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 33471 } }
I20260808 16:57:01.441466 17209 catalog_manager.cc:5495] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae940e53ec88472ba06e15f8e4b95748 (127.8.14.67). New cstate: current_term: 1 leader_uuid: "ae940e53ec88472ba06e15f8e4b95748" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb7e31d9946142de9f8efa0adc06ab78" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c196200dfad6458dbbe33229e9f06d7e" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae940e53ec88472ba06e15f8e4b95748" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 33471 } health_report { overall_health: HEALTHY } } }
I20260808 16:57:01.457801 19131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.455808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56208 (local address 127.8.14.67:33471)
0808 16:57:01.455854 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.455855 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.455864 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:57:01.455928 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.455928 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.455943 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.456000 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:57:01.456113 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.456542 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.457206 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.457378 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.457426 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.457561 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:57:01.457647 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:57:01.457703 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.457735 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.457749 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 16:57:01.459405 18808 raft_consensus.cc:1261] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78 [term 1 FOLLOWER]: Refusing update from remote peer ae940e53ec88472ba06e15f8e4b95748: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:57:01.459420 18588 raft_consensus.cc:1261] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e [term 1 FOLLOWER]: Refusing update from remote peer ae940e53ec88472ba06e15f8e4b95748: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:57:01.459736 19149 consensus_queue.cc:1035] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb7e31d9946142de9f8efa0adc06ab78" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:57:01.459882 19143 consensus_queue.cc:1035] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c196200dfad6458dbbe33229e9f06d7e" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:57:01.463109 19155 mvcc.cc:205] Tried to move back new op lower bound from 7316308875095367680 to 7316308875020857344. Current Snapshot: MvccSnapshot[applied={T|T < 7316308875095367680}]
I20260808 16:57:01.463723 19156 mvcc.cc:205] Tried to move back new op lower bound from 7316308875095367680 to 7316308875020857344. Current Snapshot: MvccSnapshot[applied={T|T < 7316308875095367680}]
I20260808 16:57:01.464641 19158 mvcc.cc:205] Tried to move back new op lower bound from 7316308875095367680 to 7316308875020857344. Current Snapshot: MvccSnapshot[applied={T|T < 7316308875095367680 or (T in {7316308875095367680})}]
I20260808 16:57:01.466239 17208 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:57:01.466343 17208 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:57:01.467720 17208 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1808f7e1cd1845c1a3da149aca6a718e: Sending DeleteTablet for 3 replicas of tablet 809ae4c4e89c4f9eaa5515c31db8ef7d
I20260808 16:57:01.468068 18778 tablet_service.cc:1514] Processing DeleteTablet for tablet 809ae4c4e89c4f9eaa5515c31db8ef7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:01 UTC) from {username='oryx', principal='oryx/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:52430
I20260808 16:57:01.468061 18564 tablet_service.cc:1514] Processing DeleteTablet for tablet 809ae4c4e89c4f9eaa5515c31db8ef7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:01 UTC) from {username='oryx', principal='oryx/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:58526
I20260808 16:57:01.468284 18983 tablet_service.cc:1514] Processing DeleteTablet for tablet 809ae4c4e89c4f9eaa5515c31db8ef7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:01 UTC) from {username='oryx', principal='oryx/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:56206
I20260808 16:57:01.468310 19170 tablet_replica.cc:330] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78: stopping tablet replica
I20260808 16:57:01.468430 19170 raft_consensus.cc:2229] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:57:01.468547 19170 raft_consensus.cc:2258] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:01.468549 19171 tablet_replica.cc:330] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748: stopping tablet replica
I20260808 16:57:01.468691 19171 raft_consensus.cc:2229] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:57:01.468858 19171 raft_consensus.cc:2258] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:01.469041 19170 ts_tablet_manager.cc:1905] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:01.469422 19171 ts_tablet_manager.cc:1905] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:01.469895 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 17168
I20260808 16:57:01.473616 19171 ts_tablet_manager.cc:1918] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:57:01.473785 19171 log.cc:1196] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/809ae4c4e89c4f9eaa5515c31db8ef7d
I20260808 16:57:01.474234 19171 ts_tablet_manager.cc:1939] T 809ae4c4e89c4f9eaa5515c31db8ef7d P ae940e53ec88472ba06e15f8e4b95748: Deleting consensus metadata
I20260808 16:57:01.477720 19168 tablet_replica.cc:330] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e: stopping tablet replica
I20260808 16:57:01.477867 19168 raft_consensus.cc:2229] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:57:01.478004 19168 raft_consensus.cc:2258] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:01.478775 19170 ts_tablet_manager.cc:1918] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:57:01.478886 19170 log.cc:1196] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/809ae4c4e89c4f9eaa5515c31db8ef7d
I20260808 16:57:01.479180 19170 ts_tablet_manager.cc:1939] T 809ae4c4e89c4f9eaa5515c31db8ef7d P cb7e31d9946142de9f8efa0adc06ab78: Deleting consensus metadata
I20260808 16:57:01.479902 19168 ts_tablet_manager.cc:1905] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:01.481088 19168 ts_tablet_manager.cc:1918] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:57:01.481194 19168 log.cc:1196] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0/minicluster-data/ts-0/wal/wals/809ae4c4e89c4f9eaa5515c31db8ef7d
I20260808 16:57:01.481462 19168 ts_tablet_manager.cc:1939] T 809ae4c4e89c4f9eaa5515c31db8ef7d P c196200dfad6458dbbe33229e9f06d7e: Deleting consensus metadata
W20260808 16:57:01.484614 18916 connection.cc:535] server connection from 127.0.0.1:56206 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 16:57:01.484632 18492 connection.cc:535] server connection from 127.0.0.1:58526 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 16:57:01.484879 18699 connection.cc:535] server connection from 127.0.0.1:52430 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 16:57:01.492023 18671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.469488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56882 (local address 127.8.14.124:46491)
0808 16:57:01.469532 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.469535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.483183 (+ 13648us) server_negotiation.cc:407] Connection header received
0808 16:57:01.483287 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.483289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.483309 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.483364 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:01.483991 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.484498 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.485683 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.486034 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.486106 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.488400 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:01.488401 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:01.488402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:01.488409 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:01.490101 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.490491 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.490492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.490492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.490547 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.491196 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.491197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.491198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.491337 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:01.491435 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.491845 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.491922 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:57:01.495451 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.486683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56892 (local address 127.8.14.124:46491)
0808 16:57:01.486728 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.486730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.487154 (+ 424us) server_negotiation.cc:407] Connection header received
0808 16:57:01.487294 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.487295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.487308 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.487349 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:01.488259 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.489011 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.490112 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.490416 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.490456 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.492206 (+ 1750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:01.492207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:01.492207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:01.492214 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:01.494010 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.494428 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.494429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.494430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.494487 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.494737 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.494738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.494738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.494933 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:01.495026 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.495234 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.495330 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:57:01.495508 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:01.486828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35768 (local address 127.8.14.125:35539)
0808 16:57:01.486877 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:01.486879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:01.487271 (+ 392us) server_negotiation.cc:407] Connection header received
0808 16:57:01.487384 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:01.487385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:01.487397 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:01.487429 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:01.488259 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.489000 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.490112 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:01.490416 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:01.490456 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:01.492474 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:01.492475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:01.492476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:01.492483 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:01.494163 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.494558 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.494559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.494560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.494620 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:01.494855 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:01.494856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:01.494856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:01.495013 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:01.495082 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:01.495335 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 16:57:01.495415 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:57:02.463989 19186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:02.463625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:53097)
0808 16:57:02.463822 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:02.463915 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":79,"threads_started":1}
I20260808 16:57:02.464118 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:02.463826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:55211)
0808 16:57:02.463998 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:02.464070 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":77,"threads_started":1}
W20260808 16:57:02.464331 19091 heartbeater.cc:644] Failed to heartbeat to 127.8.14.126:42179 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
W20260808 16:57:02.464636 18664 heartbeater.cc:644] Failed to heartbeat to 127.8.14.126:42179 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
I20260808 16:57:02.464964 19190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:02.464615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:45617)
0808 16:57:02.464830 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:02.464902 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":49,"threads_started":1}
W20260808 16:57:02.465193 18875 heartbeater.cc:644] Failed to heartbeat to 127.8.14.126:42179 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
I20260808 16:57:02.968187 19191 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1808f7e1cd1845c1a3da149aca6a718e)
I20260808 16:57:02.968327 19191 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: false } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } }
I20260808 16:57:02.968549 19191 leader_election.cc:288] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1808f7e1cd1845c1a3da149aca6a718e (127.8.14.126:42179)
I20260808 16:57:02.968940 19194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:02.968688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:60420)
0808 16:57:02.968847 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:02.968901 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":72,"threads_started":1}
W20260808 16:57:02.969079 18098 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
I20260808 16:57:02.969455 19194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:02.969354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:60428)
0808 16:57:02.969405 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:02.969431 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 16:57:02.969552 18098 leader_election.cc:334] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1808f7e1cd1845c1a3da149aca6a718e (127.8.14.126:42179): Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
I20260808 16:57:02.969594 18098 leader_election.cc:302] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: a2cc6282c44644d4b7a4f199a38d6bc8; no voters: 1808f7e1cd1845c1a3da149aca6a718e
I20260808 16:57:02.969676 19191 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:57:03.465236 19197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:03.464832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:33475)
0808 16:57:03.464991 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:03.465175 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260808 16:57:03.465642 19199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:03.464953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:54973)
0808 16:57:03.465561 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:03.465597 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":71,"threads_started":1}
I20260808 16:57:03.465780 19200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:03.465496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:60463)
0808 16:57:03.465656 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:03.465728 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":50,"threads_started":1}
I20260808 16:57:04.466442 19203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:04.466150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:36311)
0808 16:57:04.466331 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:04.466379 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":59,"threads_started":1}
I20260808 16:57:04.466521 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:04.466222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:60149)
0808 16:57:04.466424 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:04.466464 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":50,"threads_started":1}
I20260808 16:57:04.466806 19202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:04.466000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:53837)
0808 16:57:04.466732 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:04.466772 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":70,"threads_started":1}
I20260808 16:57:04.874886 19206 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1808f7e1cd1845c1a3da149aca6a718e)
I20260808 16:57:04.875047 19206 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: false } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } }
I20260808 16:57:04.875252 19206 leader_election.cc:288] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1808f7e1cd1845c1a3da149aca6a718e (127.8.14.126:42179)
I20260808 16:57:04.875751 19208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:04.875553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:60704)
0808 16:57:04.875676 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:04.875712 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":72,"threads_started":1}
I20260808 16:57:04.876166 19208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:04.876089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:60708)
0808 16:57:04.876120 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:04.876148 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260808 16:57:04.876248 18098 leader_election.cc:334] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1808f7e1cd1845c1a3da149aca6a718e (127.8.14.126:42179): Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
I20260808 16:57:04.876348 18098 leader_election.cc:302] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: a2cc6282c44644d4b7a4f199a38d6bc8; no voters: 1808f7e1cd1845c1a3da149aca6a718e
I20260808 16:57:04.876444 19206 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:57:05.467480 19212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:05.467232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:37109)
0808 16:57:05.467418 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:05.467443 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":59,"threads_started":1}
I20260808 16:57:05.467468 19213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:05.467256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:45857)
0808 16:57:05.467365 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:05.467411 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":50,"threads_started":1}
I20260808 16:57:05.467514 19211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:05.467183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:53729)
0808 16:57:05.467421 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:05.467460 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":74,"threads_started":1}
I20260808 16:57:06.468221 19214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:06.467926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:47583)
0808 16:57:06.468107 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:06.468148 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
I20260808 16:57:06.468484 19215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:06.468189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:37183)
0808 16:57:06.468411 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:06.468449 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":62,"threads_started":1}
I20260808 16:57:06.468648 19216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:06.468411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:46069)
0808 16:57:06.468563 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:06.468598 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":66,"threads_started":1}
I20260808 16:57:07.469161 19218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:07.468907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:55255)
0808 16:57:07.469071 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:07.469109 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":69,"threads_started":1}
I20260808 16:57:07.469161 19217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:07.468885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:50669)
0808 16:57:07.469074 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:07.469109 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":70,"threads_started":1}
I20260808 16:57:07.469301 19221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:07.469065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:52689)
0808 16:57:07.469208 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:07.469248 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260808 16:57:08.152923 19223 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1808f7e1cd1845c1a3da149aca6a718e)
I20260808 16:57:08.153034 19223 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: false } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } }
I20260808 16:57:08.153213 19223 leader_election.cc:288] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1808f7e1cd1845c1a3da149aca6a718e (127.8.14.126:42179)
I20260808 16:57:08.153532 19226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:08.153351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:35254)
0808 16:57:08.153446 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:08.153489 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":45,"threads_started":1}
W20260808 16:57:08.153658 18098 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260808 16:57:08.154115 19226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:08.154013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:35266)
0808 16:57:08.154066 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:08.154095 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 16:57:08.154189 18098 leader_election.cc:334] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1808f7e1cd1845c1a3da149aca6a718e (127.8.14.126:42179): Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
I20260808 16:57:08.154253 18098 leader_election.cc:302] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: a2cc6282c44644d4b7a4f199a38d6bc8; no voters: 1808f7e1cd1845c1a3da149aca6a718e
I20260808 16:57:08.154348 19223 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:57:08.469959 19229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:08.469668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:51977)
0808 16:57:08.469852 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:08.469894 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":69,"threads_started":1}
I20260808 16:57:08.469990 19230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:08.469748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:47375)
0808 16:57:08.469894 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:08.469934 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260808 16:57:08.470088 19228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:08.469668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:59107)
0808 16:57:08.470007 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:08.470042 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":72,"threads_started":1}
I20260808 16:57:09.470755 19231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:09.470443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:38467)
0808 16:57:09.470648 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:09.470688 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":70,"threads_started":1}
I20260808 16:57:09.471026 19232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:09.470539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:46659)
0808 16:57:09.470891 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:09.470967 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":65,"threads_started":1}
I20260808 16:57:09.471069 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:09.470768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:41243)
0808 16:57:09.470991 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:09.471024 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":93,"threads_started":1}
I20260808 16:57:10.471812 19238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:10.471424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:53337)
0808 16:57:10.471699 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:10.471746 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":67,"threads_started":1}
I20260808 16:57:10.471871 19243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:10.471660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:40887)
0808 16:57:10.471779 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:10.471819 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":49,"threads_started":1}
I20260808 16:57:10.472024 19239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:10.471626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:40673)
0808 16:57:10.471945 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:10.471982 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":63,"threads_started":1}
I20260808 16:57:10.738384 19244 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1808f7e1cd1845c1a3da149aca6a718e)
I20260808 16:57:10.738603 19244 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: false } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } }
I20260808 16:57:10.738797 19244 leader_election.cc:288] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1808f7e1cd1845c1a3da149aca6a718e (127.8.14.126:42179)
I20260808 16:57:10.739182 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:10.738993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:35432)
0808 16:57:10.739101 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:10.739143 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":48,"threads_started":1}
I20260808 16:57:10.739661 19245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:10.739561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:35442)
0808 16:57:10.739611 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:10.739642 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 16:57:10.739769 18098 leader_election.cc:334] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1808f7e1cd1845c1a3da149aca6a718e (127.8.14.126:42179): Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
I20260808 16:57:10.739840 18098 leader_election.cc:302] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: a2cc6282c44644d4b7a4f199a38d6bc8; no voters: 1808f7e1cd1845c1a3da149aca6a718e
I20260808 16:57:10.739905 19244 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:57:11.472658 19249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:11.472431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:41303)
0808 16:57:11.472560 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:11.472598 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260808 16:57:11.472846 19252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:11.472479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:53799)
0808 16:57:11.472756 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:11.472808 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":62,"threads_started":1}
I20260808 16:57:11.473126 19254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:11.472936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:44415)
0808 16:57:11.473037 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:11.473080 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":58,"threads_started":1}
I20260808 16:57:12.473629 19255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:12.473234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:37523)
0808 16:57:12.473523 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:12.473571 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":76,"threads_started":1}
I20260808 16:57:12.473834 19257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:12.473616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:59569)
0808 16:57:12.473740 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:12.473782 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":52,"threads_started":1}
I20260808 16:57:12.473876 19256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:12.473503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:56569)
0808 16:57:12.473804 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:12.473832 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":57,"threads_started":1}
I20260808 16:57:13.474495 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:13.474168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:54239)
0808 16:57:13.474395 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:13.474437 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":91,"threads_started":1}
I20260808 16:57:13.474607 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:13.474419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:38283)
0808 16:57:13.474525 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:13.474557 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":58,"threads_started":1}
I20260808 16:57:13.474742 19261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:13.474454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:42363)
0808 16:57:13.474667 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:13.474698 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":57,"threads_started":1}
I20260808 16:57:14.120826 19262 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1808f7e1cd1845c1a3da149aca6a718e)
I20260808 16:57:14.120931 19262 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: false } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } }
I20260808 16:57:14.121138 19262 leader_election.cc:288] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1808f7e1cd1845c1a3da149aca6a718e (127.8.14.126:42179)
I20260808 16:57:14.121526 19263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:14.121316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:35636)
0808 16:57:14.121450 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:14.121485 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
W20260808 16:57:14.121616 18098 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260808 16:57:14.122004 19263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:14.121890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:35646)
0808 16:57:14.121946 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:14.121983 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 16:57:14.122084 18098 leader_election.cc:334] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1808f7e1cd1845c1a3da149aca6a718e (127.8.14.126:42179): Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
I20260808 16:57:14.122185 18098 leader_election.cc:302] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: a2cc6282c44644d4b7a4f199a38d6bc8; no voters: 1808f7e1cd1845c1a3da149aca6a718e
I20260808 16:57:14.122289 19262 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:57:14.475415 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:14.475137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:57017)
0808 16:57:14.475317 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:14.475358 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":75,"threads_started":1}
I20260808 16:57:14.475427 19265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:14.475090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:46265)
0808 16:57:14.475333 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:14.475371 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":100,"threads_started":1}
I20260808 16:57:14.475800 19268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:14.475295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:34617)
0808 16:57:14.475717 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:14.475754 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":67,"threads_started":1}
I20260808 16:57:15.476279 19271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:15.476020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:48877)
0808 16:57:15.476183 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:15.476226 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":66,"threads_started":1}
I20260808 16:57:15.476294 19273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:15.476055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:59215)
0808 16:57:15.476197 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:15.476227 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":67,"threads_started":1}
I20260808 16:57:15.476670 19274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:15.476265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:42845)
0808 16:57:15.476584 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:15.476621 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":48,"threads_started":1}
I20260808 16:57:16.477182 19276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:16.476813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:41841)
0808 16:57:16.477049 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:16.477095 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":69,"threads_started":1}
I20260808 16:57:16.477402 19275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:16.476814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:33869)
0808 16:57:16.477317 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:16.477354 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":67,"threads_started":1}
I20260808 16:57:16.477509 19278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:16.477162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:50431)
0808 16:57:16.477417 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:16.477454 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":52,"threads_started":1}
I20260808 16:57:17.478219 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:17.477918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:41115)
0808 16:57:17.478118 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:17.478165 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":85,"threads_started":1}
I20260808 16:57:17.478225 19280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:17.477919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:40835)
0808 16:57:17.478125 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:17.478167 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":67,"threads_started":1}
I20260808 16:57:17.478346 19284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:17.478131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:48541)
0808 16:57:17.478261 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:17.478289 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":46,"threads_started":1}
I20260808 16:57:18.479522 19286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:18.479110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:51683)
0808 16:57:18.479410 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:18.479457 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":59,"threads_started":1}
I20260808 16:57:18.479549 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:18.479341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:52471)
0808 16:57:18.479455 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:18.479494 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":48,"threads_started":1}
I20260808 16:57:18.479979 19285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:18.478872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:57619)
0808 16:57:18.479899 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:18.479935 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1009,"thread_start_us":89,"threads_started":1}
I20260808 16:57:19.179741 19288 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1808f7e1cd1845c1a3da149aca6a718e)
I20260808 16:57:19.179849 19288 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1808f7e1cd1845c1a3da149aca6a718e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42179 } } peers { permanent_uuid: "a2cc6282c44644d4b7a4f199a38d6bc8" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 35539 } attrs { promote: false } } peers { permanent_uuid: "abd97edeb20a4efd922778124c11f19b" member_type: NON_VOTER last_known_addr { host: "127.8.14.124" port: 46491 } attrs { promote: true } }
I20260808 16:57:19.180053 19288 leader_election.cc:288] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1808f7e1cd1845c1a3da149aca6a718e (127.8.14.126:42179)
I20260808 16:57:19.180450 19290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:19.180232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:51646)
0808 16:57:19.180362 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:19.180410 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
W20260808 16:57:19.180544 18098 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
I20260808 16:57:19.180943 19290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:19.180838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.0.0.1:51652)
0808 16:57:19.180891 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:19.180917 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 16:57:19.181030 18098 leader_election.cc:334] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1808f7e1cd1845c1a3da149aca6a718e (127.8.14.126:42179): Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
I20260808 16:57:19.181069 18098 leader_election.cc:302] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: a2cc6282c44644d4b7a4f199a38d6bc8; no voters: 1808f7e1cd1845c1a3da149aca6a718e
I20260808 16:57:19.181162 19288 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a2cc6282c44644d4b7a4f199a38d6bc8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:57:19.480261 19293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:19.480007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:50445)
0808 16:57:19.480150 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:19.480195 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260808 16:57:19.480269 19294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:19.480022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:45241)
0808 16:57:19.480173 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:19.480215 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":104,"threads_started":1}
I20260808 16:57:19.480574 19297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:19.480327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:57515)
0808 16:57:19.480494 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:19.480532 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":60,"threads_started":1}
I20260808 16:57:20.481149 19298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:20.480886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:59967)
0808 16:57:20.481033 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:20.481080 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
I20260808 16:57:20.481172 19299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:20.480886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:50579)
0808 16:57:20.481077 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:20.481120 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":82,"threads_started":1}
I20260808 16:57:20.481328 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:20.481082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:55793)
0808 16:57:20.481212 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:20.481287 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260808 16:57:21.481864 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:21.481595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:60157)
0808 16:57:21.481735 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:21.481799 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
I20260808 16:57:21.481948 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:21.481742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:45031)
0808 16:57:21.481856 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:21.481895 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
I20260808 16:57:21.482215 19308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:21.481954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:56123)
0808 16:57:21.482130 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:21.482168 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":57,"threads_started":1}
I20260808 16:57:22.482631 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:22.482368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:34541)
0808 16:57:22.482521 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:22.482567 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":71,"threads_started":1}
I20260808 16:57:22.482650 19309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:22.482362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:39471)
0808 16:57:22.482538 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:22.482586 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":67,"threads_started":1}
I20260808 16:57:22.482847 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:22.482555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:41229)
0808 16:57:22.482684 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:22.482789 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260808 16:57:23.483249 19313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:23.483000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:41761)
0808 16:57:23.483151 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:23.483195 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260808 16:57:23.483280 19314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:23.483042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:50665)
0808 16:57:23.483206 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:23.483237 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":60,"threads_started":1}
I20260808 16:57:23.483461 19316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:23.483190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:38039)
0808 16:57:23.483324 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:23.483414 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260808 16:57:24.483911 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:24.483614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:39125)
0808 16:57:24.483785 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:24.483846 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":110,"threads_started":1}
I20260808 16:57:24.484074 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:24.483617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:49595)
0808 16:57:24.483991 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:24.484028 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":68,"threads_started":1}
I20260808 16:57:24.484143 19320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:24.483876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:48135)
0808 16:57:24.484036 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:24.484080 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":49,"threads_started":1}
I20260808 16:57:25.484752 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:25.484504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:35789)
0808 16:57:25.484648 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:25.484692 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":66,"threads_started":1}
I20260808 16:57:25.484778 19324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:25.484600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:40021)
0808 16:57:25.484705 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:25.484736 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":53,"threads_started":1}
I20260808 16:57:25.485002 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:25.484460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:38047)
0808 16:57:25.484921 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:25.484954 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":73,"threads_started":1}
I20260808 16:57:26.485802 19327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:26.485541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:44167)
0808 16:57:26.485670 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:26.485714 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260808 16:57:26.485881 19326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:26.485232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:56663)
0808 16:57:26.485791 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:26.485834 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":98,"threads_started":1}
I20260808 16:57:26.485881 19328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:26.485549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:54583)
0808 16:57:26.485809 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:26.485839 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":60,"threads_started":1}
I20260808 16:57:27.486593 19330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:27.486229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:43777)
0808 16:57:27.486489 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:27.486532 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":74,"threads_started":1}
I20260808 16:57:27.486668 19332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:27.486311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:55923)
0808 16:57:27.486583 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:27.486616 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":59,"threads_started":1}
I20260808 16:57:27.486959 19331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:27.486310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:51627)
0808 16:57:27.486859 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:27.486893 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":62,"threads_started":1}
I20260808 16:57:28.487362 19333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:28.487101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:38945)
0808 16:57:28.487237 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:28.487299 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":73,"threads_started":1}
I20260808 16:57:28.487435 19334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:28.487183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:53289)
0808 16:57:28.487337 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:28.487384 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":61,"threads_started":1}
I20260808 16:57:28.487766 19335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:28.487526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:51111)
0808 16:57:28.487682 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:28.487719 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":48,"threads_started":1}
I20260808 16:57:29.488509 19336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:29.488227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:36415)
0808 16:57:29.488403 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:29.488445 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":68,"threads_started":1}
I20260808 16:57:29.488770 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:29.488362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:60793)
0808 16:57:29.488687 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:29.488722 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":59,"threads_started":1}
I20260808 16:57:29.488970 19339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:29.488227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:46981)
0808 16:57:29.488893 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:29.488930 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":71,"threads_started":1}
I20260808 16:57:30.489732 19341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:30.489295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:39021)
0808 16:57:30.489636 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:30.489680 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":65,"threads_started":1}
I20260808 16:57:30.489792 19342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:30.489521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:43375)
0808 16:57:30.489722 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:30.489753 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":53,"threads_started":1}
I20260808 16:57:30.489792 19343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:30.489547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:42591)
0808 16:57:30.489707 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:30.489746 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":61,"threads_started":1}
I20260808 16:57:31.490504 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:31.490245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.65:53199)
0808 16:57:31.490407 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:31.490449 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":63,"threads_started":1}
I20260808 16:57:31.490541 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:31.490218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.66:53339)
0808 16:57:31.490453 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:31.490493 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":103,"threads_started":1}
I20260808 16:57:31.490593 19344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:31.490218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42179 (local address 127.8.14.67:51567)
0808 16:57:31.490477 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:31.490508 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:42179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":63,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:903: Failure
Failed
Bad status: Timed out: Could not connect to the cluster: ConnectToClusterRpc(addrs: 127.8.14.126:42179,127.8.14.125:35539,127.8.14.124:46491, num_attempts: 243) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.8.14.126:42179,127.8.14.125:35539,127.8.14.124:46491, num_attempts: 1)
I20260808 16:57:31.678794 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 18328
I20260808 16:57:31.692399 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 18677
I20260808 16:57:31.704198 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 18888
I20260808 16:57:31.717319 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 17918
I20260808 16:57:31.726190 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 18197
2026-08-08T16:57:31Z chronyd exiting
I20260808 16:57:31.743969 8249 test_util.cc:175] -----------------------------------------------
I20260808 16:57:31.744047 8249 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786208190438830-8249-0
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (33313 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:57:31 dist-test-slave-1nl6 krb5kdc[19357](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:57:31 dist-test-slave-1nl6 krb5kdc[19357](info): set up 2 sockets
Aug 08 16:57:31 dist-test-slave-1nl6 krb5kdc[19357](info): commencing operation
krb5kdc: starting...
W20260808 16:57:32.943222 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:57:33 dist-test-slave-1nl6 krb5kdc[19357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:57:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:57:33Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.126.keytab.
Entry for principal oryx/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.126.keytab.
I20260808 16:57:33.063815 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:40721
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.126.keytab
--principal=oryx/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:44847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:40721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:33.149926 19378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:33.150153 19378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:33.150192 19378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:33.151409 19378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:57:33.151468 19378 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 16:57:33.151490 19378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:33.151525 19378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:33.151552 19378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:57:33.151576 19378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:57:33.153056 19378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:44847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:40721
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.126.keytab
--principal=oryx/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:40721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:33.153333 19378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:33.153553 19378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:33.158428 19390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:33.158402 19385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:33.158411 19387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:33.158456 19378 server_base.cc:1034] running on GCE node
I20260808 16:57:33.158885 19378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:33.159197 19378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:33.160367 19378 hybrid_clock.cc:648] HybridClock initialized: now 1786208253160360 us; error 44 us; skew 500 ppm
Aug 08 16:57:33 dist-test-slave-1nl6 krb5kdc[19357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208253, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:33.162889 19378 init.cc:377] Logged in from keytab as oryx/127.8.14.126@KRBTEST.COM (short username oryx)
I20260808 16:57:33.163393 19378 webserver.cc:460] Webserver started at http://127.8.14.126:39303/ using document root <none> and password file <none>
I20260808 16:57:33.163611 19378 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:33.163686 19378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:33.163849 19378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:33.164911 19378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "2d0a853d2caa40f38e61c2e2fdb1f2e8"
format_stamp: "Formatted at 2026-08-08 16:57:33 on dist-test-slave-1nl6"
server_key: "20f727ee22d39eaa968b219a962b14f0"
server_key_iv: "0cb4480f133e19b3a5942983faef57ee"
server_key_version: "encryptionkey@0"
I20260808 16:57:33.165279 19378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "2d0a853d2caa40f38e61c2e2fdb1f2e8"
format_stamp: "Formatted at 2026-08-08 16:57:33 on dist-test-slave-1nl6"
server_key: "20f727ee22d39eaa968b219a962b14f0"
server_key_iv: "0cb4480f133e19b3a5942983faef57ee"
server_key_version: "encryptionkey@0"
I20260808 16:57:33.166623 19378 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:57:33.167419 19399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:33.167589 19378 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:33.167709 19378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "2d0a853d2caa40f38e61c2e2fdb1f2e8"
format_stamp: "Formatted at 2026-08-08 16:57:33 on dist-test-slave-1nl6"
server_key: "20f727ee22d39eaa968b219a962b14f0"
server_key_iv: "0cb4480f133e19b3a5942983faef57ee"
server_key_version: "encryptionkey@0"
I20260808 16:57:33.167786 19378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:33.177412 19378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:33.179669 19378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:33.179826 19378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:33.184695 19378 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:40721
I20260808 16:57:33.184742 19503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:40721 every 8 connection(s)
I20260808 16:57:33.185050 19378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:57:33.185631 19504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:33.187970 19504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8: Bootstrap starting.
I20260808 16:57:33.188508 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 19378
I20260808 16:57:33.188622 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:57:33.188699 19504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:33.188838 8249 external_mini_cluster.cc:1428] Setting key 0add0dc408f9b480bca10bb0bc013eda
I20260808 16:57:33.189059 19504 log.cc:824] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8: Log is configured to *not* fsync() on all Append() calls
I20260808 16:57:33.189916 19504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8: No bootstrap required, opened a new log
I20260808 16:57:33.191787 19504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0a853d2caa40f38e61c2e2fdb1f2e8" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40721 } }
I20260808 16:57:33.191928 19504 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:33.191951 19504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d0a853d2caa40f38e61c2e2fdb1f2e8, State: Initialized, Role: FOLLOWER
I20260808 16:57:33.192083 19504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0a853d2caa40f38e61c2e2fdb1f2e8" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40721 } }
I20260808 16:57:33.192170 19504 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:57:33.192193 19504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:57:33.192263 19504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:33.192863 19504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0a853d2caa40f38e61c2e2fdb1f2e8" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40721 } }
I20260808 16:57:33.192981 19504 leader_election.cc:302] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d0a853d2caa40f38e61c2e2fdb1f2e8; no voters:
I20260808 16:57:33.193146 19504 leader_election.cc:288] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:57:33.193221 19514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:57:33.193368 19514 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [term 1 LEADER]: Becoming Leader. State: Replica: 2d0a853d2caa40f38e61c2e2fdb1f2e8, State: Running, Role: LEADER
I20260808 16:57:33.193380 19504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:57:33.194589 19514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0a853d2caa40f38e61c2e2fdb1f2e8" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40721 } }
I20260808 16:57:33.195355 19516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d0a853d2caa40f38e61c2e2fdb1f2e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0a853d2caa40f38e61c2e2fdb1f2e8" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40721 } } }
I20260808 16:57:33.195353 19515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d0a853d2caa40f38e61c2e2fdb1f2e8. Latest consensus state: current_term: 1 leader_uuid: "2d0a853d2caa40f38e61c2e2fdb1f2e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0a853d2caa40f38e61c2e2fdb1f2e8" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40721 } } }
I20260808 16:57:33.195457 19516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [sys.catalog]: This master's current role is: LEADER
I20260808 16:57:33.195480 19515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8 [sys.catalog]: This master's current role is: LEADER
I20260808 16:57:33.195677 19529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:57:33.196208 19529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 16:57:33 dist-test-slave-1nl6 krb5kdc[19357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM
I20260808 16:57:33.197793 19529 catalog_manager.cc:1347] Generated new cluster ID: 6a4500199142444892fe512c61fcebaa
I20260808 16:57:33.197881 19529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:57:33.200985 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.189445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48956 (local address 127.8.14.126:40721)
0808 16:57:33.189670 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:33.189680 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:33.189719 (+ 39us) server_negotiation.cc:407] Connection header received
0808 16:57:33.190234 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:33.190242 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:33.190304 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:33.190425 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:33.191202 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.192566 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.193634 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.194130 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.194190 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.197230 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:33.197234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:33.197234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:33.197247 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:33.199354 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:33.199743 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:33.199746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:33.199746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:33.199859 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:33.200064 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:33.200065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:33.200066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:33.200386 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:33.200551 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:33.200631 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 16:57:33.200773 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":104,"threads_started":1}
I20260808 16:57:33.220548 19529 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:57:33.221060 19529 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:57:33.225082 19529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8: Generated new TSK 0
I20260808 16:57:33.225253 19529 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.65.keytab.
Entry for principal oryx/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.65.keytab.
I20260808 16:57:33.253054 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:40721
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.65.keytab
--principal=oryx/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:44847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:33.334296 19537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:33.334527 19537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:33.334569 19537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:33.335796 19537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:33.335866 19537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:33.335904 19537 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:57:33.335942 19537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:57:33.337462 19537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:44847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.65.keytab
--principal=oryx/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.14.126:40721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:33.337729 19537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:33.337960 19537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:33.342991 19551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:33.343024 19547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:33.342991 19546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:33.343314 19537 server_base.cc:1034] running on GCE node
I20260808 16:57:33.343492 19537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:33.343694 19537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:33.344861 19537 hybrid_clock.cc:648] HybridClock initialized: now 1786208253344836 us; error 40 us; skew 500 ppm
Aug 08 16:57:33 dist-test-slave-1nl6 krb5kdc[19357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208253, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:33.347466 19537 init.cc:377] Logged in from keytab as oryx/127.8.14.65@KRBTEST.COM (short username oryx)
I20260808 16:57:33.347914 19537 webserver.cc:460] Webserver started at http://127.8.14.65:41223/ using document root <none> and password file <none>
I20260808 16:57:33.348105 19537 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:33.348145 19537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:33.348258 19537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:33.349253 19537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "77b30f4dc614497fb83c6db7b38fbe88"
format_stamp: "Formatted at 2026-08-08 16:57:33 on dist-test-slave-1nl6"
server_key: "d51424a486a67db854779611d0a3da3c"
server_key_iv: "97307b9a0c3e80ab290f98f6dcc6d6cd"
server_key_version: "encryptionkey@0"
I20260808 16:57:33.349579 19537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "77b30f4dc614497fb83c6db7b38fbe88"
format_stamp: "Formatted at 2026-08-08 16:57:33 on dist-test-slave-1nl6"
server_key: "d51424a486a67db854779611d0a3da3c"
server_key_iv: "97307b9a0c3e80ab290f98f6dcc6d6cd"
server_key_version: "encryptionkey@0"
I20260808 16:57:33.350941 19537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 16:57:33.351706 19563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:33.351868 19537 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:57:33.351961 19537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "77b30f4dc614497fb83c6db7b38fbe88"
format_stamp: "Formatted at 2026-08-08 16:57:33 on dist-test-slave-1nl6"
server_key: "d51424a486a67db854779611d0a3da3c"
server_key_iv: "97307b9a0c3e80ab290f98f6dcc6d6cd"
server_key_version: "encryptionkey@0"
I20260808 16:57:33.352018 19537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:33.380656 19537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:33.383059 19537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:33.383185 19537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:33.383673 19537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:33.383718 19537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:33.383744 19537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:33.383757 19537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:33.390492 19537 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:43729
I20260808 16:57:33.390584 19748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:43729 every 8 connection(s)
Aug 08 16:57:33 dist-test-slave-1nl6 krb5kdc[19357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208253, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM
I20260808 16:57:33.390986 19537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:57:33.393934 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.384198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:43539 (local address 127.8.14.126:40721)
0808 16:57:33.384276 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:33.384278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:33.384479 (+ 201us) server_negotiation.cc:407] Connection header received
0808 16:57:33.384972 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:33.384974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:33.384991 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:33.385074 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:33.385982 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.386830 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.388463 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.388795 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.388881 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.391546 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:33.391548 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:33.391548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:33.391554 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:33.392697 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:33.393061 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:33.393062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:33.393062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:33.393114 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:33.393321 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:33.393321 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:33.393321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:33.393461 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:33.393524 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:33.393755 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 16:57:33.393850 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 16:57:33.394234 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.384163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40721 (local address 127.8.14.65:43539)
0808 16:57:33.384349 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:33.384391 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:33.384876 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:33.385178 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:33.385182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:33.385573 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:33.385864 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.385872 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.386977 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.386979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.388366 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.388370 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.388886 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.388887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.388985 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.389519 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:33.389528 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:33.391460 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:33.392767 (+ 1307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:33.392773 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:33.392775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:33.392992 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:33.393156 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:33.393156 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:33.393156 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:33.393260 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:33.393527 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:33.393532 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:33.393642 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 16:57:33.393728 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 16:57:33.393909 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
I20260808 16:57:33.394766 19750 heartbeater.cc:347] Connected to a master server at 127.8.14.126:40721
I20260808 16:57:33.394862 19750 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:33.395071 19750 heartbeater.cc:511] Master 127.8.14.126:40721 requested a full tablet report, sending...
I20260808 16:57:33.395720 19429 ts_manager.cc:194] Registered new tserver with Master: 77b30f4dc614497fb83c6db7b38fbe88 (127.8.14.65:43729)
I20260808 16:57:33.396564 19429 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:43539
I20260808 16:57:33.398582 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 19537
I20260808 16:57:33.398716 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:57:33.398955 8249 external_mini_cluster.cc:1428] Setting key ff3e0e8eac8c57927e5dbc3bfa89f016
I20260808 16:57:33.401618 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.395878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:47733 (local address 127.8.14.126:40721)
0808 16:57:33.395919 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:33.395921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:33.396056 (+ 135us) server_negotiation.cc:407] Connection header received
0808 16:57:33.396079 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:33.396080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:33.396090 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:33.396159 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:33.396413 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.397045 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.397748 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.397965 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.398041 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.399025 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:33.399027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:33.399027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:33.399034 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:33.400489 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:33.400846 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:33.400847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:33.400847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:33.400904 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:33.401112 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:33.401113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:33.401114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:33.401223 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:33.401296 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:33.401465 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 16:57:33.401532 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:57:33.401624 19764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.395833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40721 (local address 127.8.14.65:47733)
0808 16:57:33.395964 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:33.395974 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:33.396044 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:33.396162 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:33.396162 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:33.396309 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:33.396353 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.396354 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.397154 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.397155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.397662 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.397664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.398042 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.398043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.398117 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.398540 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:33.398545 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:33.398953 (+ 408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:33.400596 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:33.400598 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:33.400598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:33.400770 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:33.400981 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:33.400982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:33.400982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:33.401041 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:33.401301 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:33.401302 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:33.401347 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 16:57:33.401426 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 16:57:33.401531 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for oryx/127.8.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.66.keytab.
Entry for principal oryx/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.66.keytab.
I20260808 16:57:33.425256 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:40721
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.66.keytab
--principal=oryx/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:44847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:33.509013 19768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:33.509208 19768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:33.509232 19768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:33.510389 19768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:33.510429 19768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:33.510449 19768 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:57:33.510468 19768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:57:33.511962 19768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:44847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.66.keytab
--principal=oryx/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.14.126:40721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:33.512157 19768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:33.512332 19768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:33.517324 19780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:33.517306 19775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:33.517472 19768 server_base.cc:1034] running on GCE node
W20260808 16:57:33.517304 19776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:33.517700 19768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:33.517966 19768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:33.519125 19768 hybrid_clock.cc:648] HybridClock initialized: now 1786208253519096 us; error 47 us; skew 500 ppm
Aug 08 16:57:33 dist-test-slave-1nl6 krb5kdc[19357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208253, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:33.526532 19768 init.cc:377] Logged in from keytab as oryx/127.8.14.66@KRBTEST.COM (short username oryx)
I20260808 16:57:33.527077 19768 webserver.cc:460] Webserver started at http://127.8.14.66:34721/ using document root <none> and password file <none>
I20260808 16:57:33.527318 19768 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:33.527387 19768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:33.527524 19768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:33.528564 19768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "c9f3e1c71eef49bca554d55fa2601dce"
format_stamp: "Formatted at 2026-08-08 16:57:33 on dist-test-slave-1nl6"
server_key: "bf852da183918bf375254447477c5fe9"
server_key_iv: "06729264f2bf536f3fbed1166e54fff6"
server_key_version: "encryptionkey@0"
I20260808 16:57:33.528945 19768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "c9f3e1c71eef49bca554d55fa2601dce"
format_stamp: "Formatted at 2026-08-08 16:57:33 on dist-test-slave-1nl6"
server_key: "bf852da183918bf375254447477c5fe9"
server_key_iv: "06729264f2bf536f3fbed1166e54fff6"
server_key_version: "encryptionkey@0"
I20260808 16:57:33.530354 19768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:57:33.531116 19794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:33.531307 19768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:57:33.531421 19768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "c9f3e1c71eef49bca554d55fa2601dce"
format_stamp: "Formatted at 2026-08-08 16:57:33 on dist-test-slave-1nl6"
server_key: "bf852da183918bf375254447477c5fe9"
server_key_iv: "06729264f2bf536f3fbed1166e54fff6"
server_key_version: "encryptionkey@0"
I20260808 16:57:33.531498 19768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:33.549759 19768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:33.551721 19768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:33.551885 19768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:33.552467 19768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:33.552529 19768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:33.552605 19768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:33.552641 19768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 16:57:33 dist-test-slave-1nl6 krb5kdc[19357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208253, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM
I20260808 16:57:33.560205 19768 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:37889
I20260808 16:57:33.560266 19981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:37889 every 8 connection(s)
I20260808 16:57:33.560655 19768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:57:33.562551 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.552870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:59341 (local address 127.8.14.126:40721)
0808 16:57:33.552941 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:33.552944 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:33.553221 (+ 277us) server_negotiation.cc:407] Connection header received
0808 16:57:33.553992 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:33.553994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:33.554005 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:33.554094 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:33.555041 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.555770 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.556898 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.557133 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.557243 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.559890 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:33.559892 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:33.559893 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:33.559901 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:33.561458 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:33.561805 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:33.561806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:33.561806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:33.561856 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:33.562007 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:33.562007 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:33.562008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:33.562097 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:33.562141 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:33.562430 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 16:57:33.562489 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 16:57:33.562772 19808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.552871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40721 (local address 127.8.14.66:59341)
0808 16:57:33.553063 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:33.553113 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:33.553862 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:33.554195 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:33.554200 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:33.554594 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:33.554948 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.554956 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.555890 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.555891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.556789 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.556792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.557259 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.557260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.557435 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.557979 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:33.557989 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:33.559785 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:33.561528 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:33.561533 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:33.561535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:33.561745 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:33.561896 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:33.561896 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:33.561897 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:33.561974 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:33.562179 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:33.562198 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:33.562308 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 16:57:33.562401 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 16:57:33.562515 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
I20260808 16:57:33.563301 19984 heartbeater.cc:347] Connected to a master server at 127.8.14.126:40721
I20260808 16:57:33.563390 19984 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:33.563573 19984 heartbeater.cc:511] Master 127.8.14.126:40721 requested a full tablet report, sending...
I20260808 16:57:33.564000 19429 ts_manager.cc:194] Registered new tserver with Master: c9f3e1c71eef49bca554d55fa2601dce (127.8.14.66:37889)
I20260808 16:57:33.564532 19429 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:59341
I20260808 16:57:33.569465 19992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.564334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40721 (local address 127.8.14.66:45191)
0808 16:57:33.564456 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:33.564467 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:33.564526 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:33.564669 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:33.564669 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:33.564839 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:33.564888 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.564890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.565522 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.565523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.566021 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.566022 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.566385 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.566386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.566456 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.566811 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:33.566814 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:33.567132 (+ 318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:33.568391 (+ 1259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:33.568392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:33.568392 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:33.568563 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:33.568780 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:33.568781 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:33.568781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:33.568843 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:33.569135 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:33.569136 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:33.569181 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 16:57:33.569261 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 16:57:33.569371 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":66,"threads_started":1}
I20260808 16:57:33.569456 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.564409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:45191 (local address 127.8.14.126:40721)
0808 16:57:33.564450 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:33.564452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:33.564510 (+ 58us) server_negotiation.cc:407] Connection header received
0808 16:57:33.564584 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:33.564585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:33.564595 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:33.564646 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:33.564978 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.565427 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.566107 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.566310 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.566385 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.567232 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:33.567234 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:33.567234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:33.567241 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:33.568306 (+ 1065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:33.568652 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:33.568653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:33.568654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:33.568711 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:33.568909 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:33.568910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:33.568910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:33.569055 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:33.569131 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:33.569301 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 16:57:33.569371 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:57:33.570900 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 19768
I20260808 16:57:33.571056 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:57:33.571355 8249 external_mini_cluster.cc:1428] Setting key 95af078ba9bba1d95f0f6e6d6d5675c3
WARNING: no policy specified for oryx/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.67.keytab.
Entry for principal oryx/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.67.keytab.
I20260808 16:57:33.596060 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:40721
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.67.keytab
--principal=oryx/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:44847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:33.678923 19997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:33.679160 19997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:33.679198 19997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:33.680342 19997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:33.680408 19997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:33.680441 19997 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 16:57:33.680480 19997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:57:33.681973 19997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:44847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.67.keytab
--principal=oryx/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.14.126:40721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:33.682229 19997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:33.682471 19997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:33.687386 20009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:33.687400 20006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:33.687434 19997 server_base.cc:1034] running on GCE node
W20260808 16:57:33.687407 20004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:33.687809 19997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:33.688012 19997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:33.689153 19997 hybrid_clock.cc:648] HybridClock initialized: now 1786208253689132 us; error 36 us; skew 500 ppm
Aug 08 16:57:33 dist-test-slave-1nl6 krb5kdc[19357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208253, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:33.691601 19997 init.cc:377] Logged in from keytab as oryx/127.8.14.67@KRBTEST.COM (short username oryx)
I20260808 16:57:33.692056 19997 webserver.cc:460] Webserver started at http://127.8.14.67:39271/ using document root <none> and password file <none>
I20260808 16:57:33.692323 19997 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:33.692395 19997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:33.692543 19997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:33.693634 19997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "0075aee99ed24683a938b7f6057b4cb0"
format_stamp: "Formatted at 2026-08-08 16:57:33 on dist-test-slave-1nl6"
server_key: "c4c18aa021426b545e9c087d92c36570"
server_key_iv: "8f2e7b6635029c793a4b1935c36c8b6c"
server_key_version: "encryptionkey@0"
I20260808 16:57:33.694046 19997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "0075aee99ed24683a938b7f6057b4cb0"
format_stamp: "Formatted at 2026-08-08 16:57:33 on dist-test-slave-1nl6"
server_key: "c4c18aa021426b545e9c087d92c36570"
server_key_iv: "8f2e7b6635029c793a4b1935c36c8b6c"
server_key_version: "encryptionkey@0"
I20260808 16:57:33.695533 19997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:57:33.696419 20019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:33.696611 19997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:57:33.696718 19997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "0075aee99ed24683a938b7f6057b4cb0"
format_stamp: "Formatted at 2026-08-08 16:57:33 on dist-test-slave-1nl6"
server_key: "c4c18aa021426b545e9c087d92c36570"
server_key_iv: "8f2e7b6635029c793a4b1935c36c8b6c"
server_key_version: "encryptionkey@0"
I20260808 16:57:33.696794 19997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:33.715835 19997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:33.717742 19997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:33.717919 19997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:33.718438 19997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:33.718506 19997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:33.718551 19997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:33.718581 19997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 16:57:33 dist-test-slave-1nl6 krb5kdc[19357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208253, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM
I20260808 16:57:33.725591 20196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:35341 every 8 connection(s)
I20260808 16:57:33.725512 19997 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:35341
I20260808 16:57:33.726290 19997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 16:57:33.728703 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.719037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:36343 (local address 127.8.14.126:40721)
0808 16:57:33.719083 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:33.719085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:33.719306 (+ 221us) server_negotiation.cc:407] Connection header received
0808 16:57:33.719841 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:33.719842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:33.719855 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:33.719909 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:33.720862 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.721712 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.723236 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.723454 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.723529 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.725912 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:33.725914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:33.725914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:33.725937 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:33.727597 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:33.727960 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:33.727961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:33.727961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:33.728012 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:33.728202 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:33.728202 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:33.728202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:33.728301 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:33.728353 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:33.728578 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 16:57:33.728639 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 16:57:33.728847 20030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.719020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40721 (local address 127.8.14.67:36343)
0808 16:57:33.719176 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:33.719217 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:33.719730 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:33.719990 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:33.719994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:33.720384 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:33.720694 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.720701 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.721829 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.721830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.723129 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.723132 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.723557 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.723559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.723683 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.724162 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:33.724170 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:33.725825 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:33.727667 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:33.727671 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:33.727672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:33.727893 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:33.728053 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:33.728053 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:33.728053 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:33.728136 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:33.728362 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:33.728366 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:33.728472 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 16:57:33.728551 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 16:57:33.728664 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260808 16:57:33.729387 20197 heartbeater.cc:347] Connected to a master server at 127.8.14.126:40721
I20260808 16:57:33.729462 20197 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:33.729656 20197 heartbeater.cc:511] Master 127.8.14.126:40721 requested a full tablet report, sending...
I20260808 16:57:33.730413 19429 ts_manager.cc:194] Registered new tserver with Master: 0075aee99ed24683a938b7f6057b4cb0 (127.8.14.67:35341)
I20260808 16:57:33.730958 19429 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:36343
I20260808 16:57:33.731002 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 19997
I20260808 16:57:33.731091 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:57:33.731279 8249 external_mini_cluster.cc:1428] Setting key eeeba08a0b68417e74b62257b8e94f5a
I20260808 16:57:33.732050 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:57:33.736184 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.730366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:52701 (local address 127.8.14.126:40721)
0808 16:57:33.730430 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:33.730432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:33.730541 (+ 109us) server_negotiation.cc:407] Connection header received
0808 16:57:33.730610 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:33.730611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:33.730621 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:33.730691 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:33.730986 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.731688 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.732433 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.732744 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.732821 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.733634 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:33.733635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:33.733636 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:33.733643 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:33.735109 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:33.735526 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:33.735527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:33.735527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:33.735589 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:33.735796 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:33.735796 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:33.735796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:33.735880 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:33.735924 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:33.736080 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 16:57:33.736132 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 16:57:33.736313 20210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.730373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40721 (local address 127.8.14.67:52701)
0808 16:57:33.730484 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:33.730495 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:33.730551 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:33.730692 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:33.730693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:33.730836 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:33.730882 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.730883 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.731809 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.731810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.732355 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.732357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.732827 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.732827 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.732874 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.733280 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:33.733291 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:33.733547 (+ 256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:33.735186 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:33.735187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:33.735188 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:33.735424 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:33.735659 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:33.735660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:33.735660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:33.735732 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:33.735939 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:33.735940 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:33.735987 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 16:57:33.736063 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 16:57:33.736171 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":64,"threads_started":1}
Aug 08 16:57:33 dist-test-slave-1nl6 krb5kdc[19357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208253, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 16:57:33 dist-test-slave-1nl6 krb5kdc[19357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:33 dist-test-slave-1nl6 krb5kdc[19357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 16:57:33.755532 19513 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:33.749951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48960 (local address 127.8.14.126:40721)
0808 16:57:33.750020 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:33.750022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:33.750031 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:57:33.750053 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:33.750054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:33.750063 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:33.750113 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:33.751012 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.751515 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.752238 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.752413 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.752468 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.755439 (+ 2971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48960: BlockingRecv error: recv got EOF from 127.0.0.1:48960 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 16:57:33 dist-test-slave-1nl6 krb5kdc[19357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208253, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM
I20260808 16:57:33.762679 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.756380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48964 (local address 127.8.14.126:40721)
0808 16:57:33.756440 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:33.756442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:33.756453 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:57:33.756539 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:33.756540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:33.756550 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:33.756606 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:33.757056 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.757510 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.758167 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.758367 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.758443 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.760444 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:33.760445 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:33.760445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:33.760451 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:33.761810 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:33.762089 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:33.762090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:33.762090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:33.762139 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:33.762291 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:33.762292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:33.762292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:33.762377 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:33.762446 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:33.762525 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 16:57:33.762613 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 16:57:33.764326 19429 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:57:33.765955 19429 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:57:33.772378 20242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.768199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:37889 (local address 127.0.0.1:38382)
0808 16:57:33.768771 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:33.768780 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:33.768855 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:33.769117 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:33.769118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:33.769120 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:33.769161 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.769162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.770394 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.770395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.771323 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.771325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.772032 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.772033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.772129 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.772145 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 16:57:33.772245 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 16:57:33.772298 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":2,"spinlock_wait_cycles":768,"thread_start_us":117,"threads_started":1}
I20260808 16:57:33.772392 20250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.768456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38382 (local address 127.8.14.66:37889)
0808 16:57:33.768648 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:33.768655 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:33.768827 (+ 172us) server_negotiation.cc:407] Connection header received
0808 16:57:33.768956 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:33.768959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:33.768974 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:33.769027 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:33.769248 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.770309 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.771425 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.771911 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.772027 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.772103 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:33.772262 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 16:57:33.772299 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":85,"threads_started":1}
I20260808 16:57:33.772660 20241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.768192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:35341 (local address 127.0.0.1:42692)
0808 16:57:33.768449 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:33.768486 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:33.768583 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:33.768764 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:33.768767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:33.768769 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:33.768848 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.768850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.770152 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.770153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.771327 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.771331 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.772386 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.772387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.772488 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.772493 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:57:33.772546 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 16:57:33.772587 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":3,"thread_start_us":59,"threads_started":1}
I20260808 16:57:33.772759 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.768195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:43729 (local address 127.0.0.1:35258)
0808 16:57:33.768624 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:33.768636 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:33.768694 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:33.768959 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:33.768960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:33.768962 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:33.769017 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.769019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.769939 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.769940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.771331 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.771332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.772493 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.772494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.772578 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.772582 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:57:33.772642 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 16:57:33.772682 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":2,"thread_start_us":93,"threads_started":1}
I20260808 16:57:33.772902 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.768457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35258 (local address 127.8.14.65:43729)
0808 16:57:33.768721 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:33.768724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:33.768734 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:57:33.768776 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:33.768780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:33.768812 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:33.768891 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:33.769092 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.769849 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.771450 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.772195 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.772270 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.772349 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:33.772660 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 16:57:33.772788 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":81,"threads_started":1}
I20260808 16:57:33.773061 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.768271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42692 (local address 127.8.14.67:35341)
0808 16:57:33.768426 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:33.768432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:33.768564 (+ 132us) server_negotiation.cc:407] Connection header received
0808 16:57:33.768666 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:33.768668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:33.768683 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:33.768757 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:33.768939 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.770067 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.771450 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.772195 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.772270 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.772345 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:33.772827 (+ 482us) server_negotiation.cc:299] Negotiation successful
0808 16:57:33.772871 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260808 16:57:33.773371 19856 tablet_service.cc:1467] Processing CreateTablet for tablet df28486bd08e405fbb520172d7cf4f24 (DEFAULT_TABLE table=test-table [id=55b7862802124ec1b6893d863680fb63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:33.773491 19643 tablet_service.cc:1467] Processing CreateTablet for tablet df28486bd08e405fbb520172d7cf4f24 (DEFAULT_TABLE table=test-table [id=55b7862802124ec1b6893d863680fb63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:33.773663 20091 tablet_service.cc:1467] Processing CreateTablet for tablet df28486bd08e405fbb520172d7cf4f24 (DEFAULT_TABLE table=test-table [id=55b7862802124ec1b6893d863680fb63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:33.773780 19856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df28486bd08e405fbb520172d7cf4f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:33.773844 19643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df28486bd08e405fbb520172d7cf4f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:33.774000 20091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df28486bd08e405fbb520172d7cf4f24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:33.777243 20251 tablet_bootstrap.cc:492] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88: Bootstrap starting.
I20260808 16:57:33.777730 20254 tablet_bootstrap.cc:492] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce: Bootstrap starting.
I20260808 16:57:33.777962 20251 tablet_bootstrap.cc:654] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:33.778131 20252 tablet_bootstrap.cc:492] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0: Bootstrap starting.
I20260808 16:57:33.778321 20251 log.cc:824] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88: Log is configured to *not* fsync() on all Append() calls
I20260808 16:57:33.778355 20254 tablet_bootstrap.cc:654] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:33.778661 20254 log.cc:824] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce: Log is configured to *not* fsync() on all Append() calls
I20260808 16:57:33.778781 20252 tablet_bootstrap.cc:654] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:33.779150 20252 log.cc:824] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0: Log is configured to *not* fsync() on all Append() calls
I20260808 16:57:33.779222 20251 tablet_bootstrap.cc:492] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88: No bootstrap required, opened a new log
I20260808 16:57:33.779282 20251 ts_tablet_manager.cc:1397] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:57:33.779500 20254 tablet_bootstrap.cc:492] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce: No bootstrap required, opened a new log
I20260808 16:57:33.779577 20254 ts_tablet_manager.cc:1397] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:57:33.780040 20252 tablet_bootstrap.cc:492] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0: No bootstrap required, opened a new log
I20260808 16:57:33.780112 20252 ts_tablet_manager.cc:1397] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 16:57:33.781038 20254 raft_consensus.cc:348] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } }
I20260808 16:57:33.781201 20254 raft_consensus.cc:374] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:33.781245 20254 raft_consensus.cc:729] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9f3e1c71eef49bca554d55fa2601dce, State: Initialized, Role: FOLLOWER
I20260808 16:57:33.781366 20254 consensus_queue.cc:260] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } }
I20260808 16:57:33.781378 20251 raft_consensus.cc:348] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } }
I20260808 16:57:33.781585 20251 raft_consensus.cc:374] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:33.781656 20251 raft_consensus.cc:729] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77b30f4dc614497fb83c6db7b38fbe88, State: Initialized, Role: FOLLOWER
I20260808 16:57:33.781652 20254 ts_tablet_manager.cc:1428] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:57:33.781698 19984 heartbeater.cc:503] Master 127.8.14.126:40721 was elected leader, sending a full tablet report...
I20260808 16:57:33.781783 20251 consensus_queue.cc:260] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } }
I20260808 16:57:33.782061 20251 ts_tablet_manager.cc:1428] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:57:33.782200 19750 heartbeater.cc:503] Master 127.8.14.126:40721 was elected leader, sending a full tablet report...
I20260808 16:57:33.782164 20252 raft_consensus.cc:348] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } }
I20260808 16:57:33.782330 20252 raft_consensus.cc:374] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:33.782428 20252 raft_consensus.cc:729] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0075aee99ed24683a938b7f6057b4cb0, State: Initialized, Role: FOLLOWER
I20260808 16:57:33.782577 20252 consensus_queue.cc:260] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } }
I20260808 16:57:33.782939 20252 ts_tablet_manager.cc:1428] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 16:57:33.783002 20197 heartbeater.cc:503] Master 127.8.14.126:40721 was elected leader, sending a full tablet report...
W20260808 16:57:33.811800 19985 tablet.cc:2369] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:57:33.891760 19751 tablet.cc:2369] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:57:33.952106 20260 raft_consensus.cc:482] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:57:33.952302 20260 raft_consensus.cc:504] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } }
I20260808 16:57:33.952673 20260 leader_election.cc:288] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77b30f4dc614497fb83c6db7b38fbe88 (127.8.14.65:43729), 0075aee99ed24683a938b7f6057b4cb0 (127.8.14.67:35341)
I20260808 16:57:33.956676 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.952999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:45545 (local address 127.8.14.65:43729)
0808 16:57:33.953053 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:33.953056 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:33.953195 (+ 139us) server_negotiation.cc:407] Connection header received
0808 16:57:33.953223 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:33.953224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:33.953235 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:33.953288 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:33.953438 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.954174 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.955699 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.956261 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.956342 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.956350 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:33.956574 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 16:57:33.956618 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 16:57:33.956723 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.952863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:43729 (local address 127.8.14.66:45545)
0808 16:57:33.953092 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:33.953102 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:33.953160 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:33.953301 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:33.953302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:33.953304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:33.953366 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.953368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.954246 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.954247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.955609 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.955612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.956368 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.956369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.956472 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.956477 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:57:33.956548 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 16:57:33.956596 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":117,"thread_start_us":120,"threads_started":1}
I20260808 16:57:33.956734 19808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.952863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:35341 (local address 127.8.14.66:46377)
0808 16:57:33.952940 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:33.952955 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:33.953014 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:33.953230 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:33.953231 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:33.953233 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:33.953315 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.953317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.954244 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.954245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.955609 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:33.955612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.956368 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.956369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:33.956472 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.956477 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:57:33.956548 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 16:57:33.956596 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":58}
I20260808 16:57:33.956931 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.952964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:46377 (local address 127.8.14.67:35341)
0808 16:57:33.953009 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:33.953011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:33.953022 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:57:33.953113 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:33.953114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:33.953125 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:33.953208 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:33.953388 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.954148 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.955697 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.956261 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.956339 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.956349 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:33.956794 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 16:57:33.956846 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:57:33.957230 20126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df28486bd08e405fbb520172d7cf4f24" candidate_uuid: "c9f3e1c71eef49bca554d55fa2601dce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0075aee99ed24683a938b7f6057b4cb0" is_pre_election: true
I20260808 16:57:33.957357 20126 raft_consensus.cc:2454] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9f3e1c71eef49bca554d55fa2601dce in term 0.
I20260808 16:57:33.957619 19799 leader_election.cc:302] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0075aee99ed24683a938b7f6057b4cb0, c9f3e1c71eef49bca554d55fa2601dce; no voters:
I20260808 16:57:33.957662 19675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df28486bd08e405fbb520172d7cf4f24" candidate_uuid: "c9f3e1c71eef49bca554d55fa2601dce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77b30f4dc614497fb83c6db7b38fbe88" is_pre_election: true
I20260808 16:57:33.957767 20260 raft_consensus.cc:2781] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:57:33.957803 19675 raft_consensus.cc:2454] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9f3e1c71eef49bca554d55fa2601dce in term 0.
I20260808 16:57:33.957834 20260 raft_consensus.cc:482] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:57:33.957858 20260 raft_consensus.cc:3037] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:33.958838 20260 raft_consensus.cc:504] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } }
I20260808 16:57:33.959007 20260 leader_election.cc:288] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [CANDIDATE]: Term 1 election: Requested vote from peers 77b30f4dc614497fb83c6db7b38fbe88 (127.8.14.65:43729), 0075aee99ed24683a938b7f6057b4cb0 (127.8.14.67:35341)
I20260808 16:57:33.959179 19675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df28486bd08e405fbb520172d7cf4f24" candidate_uuid: "c9f3e1c71eef49bca554d55fa2601dce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77b30f4dc614497fb83c6db7b38fbe88"
I20260808 16:57:33.959277 19675 raft_consensus.cc:3037] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:33.959313 20126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df28486bd08e405fbb520172d7cf4f24" candidate_uuid: "c9f3e1c71eef49bca554d55fa2601dce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0075aee99ed24683a938b7f6057b4cb0"
I20260808 16:57:33.959398 20126 raft_consensus.cc:3037] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:33.960037 19675 raft_consensus.cc:2454] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9f3e1c71eef49bca554d55fa2601dce in term 1.
I20260808 16:57:33.960122 20126 raft_consensus.cc:2454] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9f3e1c71eef49bca554d55fa2601dce in term 1.
I20260808 16:57:33.960281 19799 leader_election.cc:302] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0075aee99ed24683a938b7f6057b4cb0, c9f3e1c71eef49bca554d55fa2601dce; no voters:
I20260808 16:57:33.960382 20260 raft_consensus.cc:2781] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:57:33.960536 20260 raft_consensus.cc:686] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [term 1 LEADER]: Becoming Leader. State: Replica: c9f3e1c71eef49bca554d55fa2601dce, State: Running, Role: LEADER
I20260808 16:57:33.960634 20260 consensus_queue.cc:237] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } }
I20260808 16:57:33.961448 19429 catalog_manager.cc:5495] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce reported cstate change: term changed from 0 to 1, leader changed from <none> to c9f3e1c71eef49bca554d55fa2601dce (127.8.14.66). New cstate: current_term: 1 leader_uuid: "c9f3e1c71eef49bca554d55fa2601dce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } health_report { overall_health: HEALTHY } } }
W20260808 16:57:33.976987 20199 tablet.cc:2369] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:57:33.999230 20250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:33.996897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38394 (local address 127.8.14.66:37889)
0808 16:57:33.996962 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:33.996968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:33.996976 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:57:33.997011 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:33.997012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:33.997031 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:33.997093 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:57:33.997218 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.997668 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.998427 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:33.998611 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:33.998665 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:33.998791 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:57:33.998938 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:57:33.999015 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:33.999125 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 16:57:33.999144 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 16:57:34.001108 20126 raft_consensus.cc:1261] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0 [term 1 FOLLOWER]: Refusing update from remote peer c9f3e1c71eef49bca554d55fa2601dce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:57:34.001111 19675 raft_consensus.cc:1261] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 FOLLOWER]: Refusing update from remote peer c9f3e1c71eef49bca554d55fa2601dce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:57:34.001468 20270 consensus_queue.cc:1035] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [LEADER]: Connected to new peer: Peer: permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:57:34.001629 20260 consensus_queue.cc:1035] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [LEADER]: Connected to new peer: Peer: permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:57:34.004247 20276 mvcc.cc:205] Tried to move back new op lower bound from 7316309008385708032 to 7316309008223158272. Current Snapshot: MvccSnapshot[applied={T|T < 7316309008385708032}]
I20260808 16:57:34.006738 19429 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:57:34.006821 19429 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:57:34.007649 19429 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8: Sending DeleteTablet for 3 replicas of tablet df28486bd08e405fbb520172d7cf4f24
I20260808 16:57:34.007968 19643 tablet_service.cc:1514] Processing DeleteTablet for tablet df28486bd08e405fbb520172d7cf4f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:34 UTC) from {username='oryx', principal='oryx/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:35258
I20260808 16:57:34.007993 19856 tablet_service.cc:1514] Processing DeleteTablet for tablet df28486bd08e405fbb520172d7cf4f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:34 UTC) from {username='oryx', principal='oryx/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:38382
I20260808 16:57:34.008042 20091 tablet_service.cc:1514] Processing DeleteTablet for tablet df28486bd08e405fbb520172d7cf4f24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:34 UTC) from {username='oryx', principal='oryx/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:42692
I20260808 16:57:34.008282 20286 tablet_replica.cc:330] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88: stopping tablet replica
I20260808 16:57:34.008428 20286 raft_consensus.cc:2229] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:57:34.008456 19429 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 16:57:34.008507 20290 tablet_replica.cc:330] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0: stopping tablet replica
W20260808 16:57:34.008589 19429 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:57:34.008584 20286 raft_consensus.cc:2258] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:34.008603 20290 raft_consensus.cc:2229] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:57:34.008708 20290 raft_consensus.cc:2258] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:34.009014 20287 tablet_replica.cc:330] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce: stopping tablet replica
I20260808 16:57:34.009177 20287 raft_consensus.cc:2229] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:57:34.009296 20290 ts_tablet_manager.cc:1905] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:34.009351 20287 raft_consensus.cc:2258] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:34.009895 20287 ts_tablet_manager.cc:1905] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:34.010442 19643 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1c73447834425d993798deeaed0956 (DEFAULT_TABLE table=test-table [id=127cfab454684266ba4510ca166c44dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:34.010562 19643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1c73447834425d993798deeaed0956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:34.011068 20290 ts_tablet_manager.cc:1918] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:57:34.011145 20290 log.cc:1196] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/df28486bd08e405fbb520172d7cf4f24
I20260808 16:57:34.011413 20290 ts_tablet_manager.cc:1939] T df28486bd08e405fbb520172d7cf4f24 P 0075aee99ed24683a938b7f6057b4cb0: Deleting consensus metadata
I20260808 16:57:34.011865 20287 ts_tablet_manager.cc:1918] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:57:34.011934 20287 log.cc:1196] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/df28486bd08e405fbb520172d7cf4f24
I20260808 16:57:34.012130 20251 tablet_bootstrap.cc:492] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88: Bootstrap starting.
I20260808 16:57:34.012168 19856 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1c73447834425d993798deeaed0956 (DEFAULT_TABLE table=test-table [id=127cfab454684266ba4510ca166c44dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:34.012243 20287 ts_tablet_manager.cc:1939] T df28486bd08e405fbb520172d7cf4f24 P c9f3e1c71eef49bca554d55fa2601dce: Deleting consensus metadata
I20260808 16:57:34.012267 19856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1c73447834425d993798deeaed0956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:34.012501 20251 tablet_bootstrap.cc:654] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:34.013180 20251 tablet_bootstrap.cc:492] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88: No bootstrap required, opened a new log
I20260808 16:57:34.013224 20251 ts_tablet_manager.cc:1397] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:57:34.013321 20254 tablet_bootstrap.cc:492] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce: Bootstrap starting.
I20260808 16:57:34.013384 20251 raft_consensus.cc:348] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.013458 20251 raft_consensus.cc:374] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:34.013478 20251 raft_consensus.cc:729] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77b30f4dc614497fb83c6db7b38fbe88, State: Initialized, Role: FOLLOWER
I20260808 16:57:34.013516 20251 consensus_queue.cc:260] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.013619 20251 ts_tablet_manager.cc:1428] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 16:57:34.013736 20254 tablet_bootstrap.cc:654] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:34.014066 19400 catalog_manager.cc:4841] TS c9f3e1c71eef49bca554d55fa2601dce (127.8.14.66:37889): tablet df28486bd08e405fbb520172d7cf4f24 (table test-table [id=55b7862802124ec1b6893d863680fb63]) successfully deleted
I20260808 16:57:34.014401 20254 tablet_bootstrap.cc:492] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce: No bootstrap required, opened a new log
I20260808 16:57:34.014410 19402 catalog_manager.cc:4841] TS 0075aee99ed24683a938b7f6057b4cb0 (127.8.14.67:35341): tablet df28486bd08e405fbb520172d7cf4f24 (table test-table [id=55b7862802124ec1b6893d863680fb63]) successfully deleted
I20260808 16:57:34.014453 20254 ts_tablet_manager.cc:1397] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:57:34.014495 20091 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1c73447834425d993798deeaed0956 (DEFAULT_TABLE table=test-table [id=127cfab454684266ba4510ca166c44dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:34.014600 20091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1c73447834425d993798deeaed0956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:34.014602 20254 raft_consensus.cc:348] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.014668 20254 raft_consensus.cc:374] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:34.014693 20254 raft_consensus.cc:729] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9f3e1c71eef49bca554d55fa2601dce, State: Initialized, Role: FOLLOWER
I20260808 16:57:34.014760 20254 consensus_queue.cc:260] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.014861 20254 ts_tablet_manager.cc:1428] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:34.015331 20286 ts_tablet_manager.cc:1905] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:34.015928 20252 tablet_bootstrap.cc:492] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0: Bootstrap starting.
I20260808 16:57:34.016321 20252 tablet_bootstrap.cc:654] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:34.016968 20286 ts_tablet_manager.cc:1918] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:57:34.017028 20286 log.cc:1196] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/wal/wals/df28486bd08e405fbb520172d7cf4f24
I20260808 16:57:34.017509 20286 ts_tablet_manager.cc:1939] T df28486bd08e405fbb520172d7cf4f24 P 77b30f4dc614497fb83c6db7b38fbe88: Deleting consensus metadata
I20260808 16:57:34.017864 19404 catalog_manager.cc:4841] TS 77b30f4dc614497fb83c6db7b38fbe88 (127.8.14.65:43729): tablet df28486bd08e405fbb520172d7cf4f24 (table test-table [id=55b7862802124ec1b6893d863680fb63]) successfully deleted
I20260808 16:57:34.018093 20252 tablet_bootstrap.cc:492] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0: No bootstrap required, opened a new log
I20260808 16:57:34.018136 20252 ts_tablet_manager.cc:1397] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:57:34.018311 20252 raft_consensus.cc:348] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.018366 20252 raft_consensus.cc:374] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:34.018380 20252 raft_consensus.cc:729] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0075aee99ed24683a938b7f6057b4cb0, State: Initialized, Role: FOLLOWER
I20260808 16:57:34.018433 20252 consensus_queue.cc:260] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.018523 20252 ts_tablet_manager.cc:1428] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:34.049212 20260 raft_consensus.cc:482] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:57:34.049342 20260 raft_consensus.cc:504] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.049500 20260 leader_election.cc:288] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77b30f4dc614497fb83c6db7b38fbe88 (127.8.14.65:43729), 0075aee99ed24683a938b7f6057b4cb0 (127.8.14.67:35341)
I20260808 16:57:34.049731 19675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1c73447834425d993798deeaed0956" candidate_uuid: "c9f3e1c71eef49bca554d55fa2601dce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77b30f4dc614497fb83c6db7b38fbe88" is_pre_election: true
I20260808 16:57:34.049732 20126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1c73447834425d993798deeaed0956" candidate_uuid: "c9f3e1c71eef49bca554d55fa2601dce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0075aee99ed24683a938b7f6057b4cb0" is_pre_election: true
I20260808 16:57:34.049839 19675 raft_consensus.cc:2454] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9f3e1c71eef49bca554d55fa2601dce in term 0.
I20260808 16:57:34.049853 20126 raft_consensus.cc:2454] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9f3e1c71eef49bca554d55fa2601dce in term 0.
I20260808 16:57:34.050020 19799 leader_election.cc:302] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0075aee99ed24683a938b7f6057b4cb0, c9f3e1c71eef49bca554d55fa2601dce; no voters:
I20260808 16:57:34.050113 20260 raft_consensus.cc:2781] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:57:34.050163 20260 raft_consensus.cc:482] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:57:34.050216 20260 raft_consensus.cc:3037] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:34.051005 20260 raft_consensus.cc:504] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.051175 20260 leader_election.cc:288] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [CANDIDATE]: Term 1 election: Requested vote from peers 77b30f4dc614497fb83c6db7b38fbe88 (127.8.14.65:43729), 0075aee99ed24683a938b7f6057b4cb0 (127.8.14.67:35341)
I20260808 16:57:34.051275 19675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1c73447834425d993798deeaed0956" candidate_uuid: "c9f3e1c71eef49bca554d55fa2601dce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77b30f4dc614497fb83c6db7b38fbe88"
I20260808 16:57:34.051352 19675 raft_consensus.cc:3037] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:34.051405 20126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1c73447834425d993798deeaed0956" candidate_uuid: "c9f3e1c71eef49bca554d55fa2601dce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0075aee99ed24683a938b7f6057b4cb0"
I20260808 16:57:34.051486 20126 raft_consensus.cc:3037] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:34.052222 19675 raft_consensus.cc:2454] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9f3e1c71eef49bca554d55fa2601dce in term 1.
I20260808 16:57:34.052230 20126 raft_consensus.cc:2454] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9f3e1c71eef49bca554d55fa2601dce in term 1.
I20260808 16:57:34.052443 19800 leader_election.cc:302] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77b30f4dc614497fb83c6db7b38fbe88, c9f3e1c71eef49bca554d55fa2601dce; no voters:
I20260808 16:57:34.052534 20260 raft_consensus.cc:2781] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:57:34.052583 20260 raft_consensus.cc:686] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [term 1 LEADER]: Becoming Leader. State: Replica: c9f3e1c71eef49bca554d55fa2601dce, State: Running, Role: LEADER
I20260808 16:57:34.052632 20260 consensus_queue.cc:237] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.053254 19429 catalog_manager.cc:5495] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce reported cstate change: term changed from 0 to 1, leader changed from <none> to c9f3e1c71eef49bca554d55fa2601dce (127.8.14.66). New cstate: current_term: 1 leader_uuid: "c9f3e1c71eef49bca554d55fa2601dce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } health_report { overall_health: UNKNOWN } } }
Aug 08 16:57:34 dist-test-slave-1nl6 krb5kdc[19357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208253, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.126@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM
I20260808 16:57:34.083873 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.075716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40721 (local address 127.0.0.1:48978)
0808 16:57:34.075906 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:34.075919 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:34.075997 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:34.076150 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:34.076151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:34.076559 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:34.076615 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:34.076618 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.077315 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.077317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:34.078158 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:34.078161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.078521 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.078522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:34.078586 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.079193 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:34.079198 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:34.080856 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:34.082490 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:34.082498 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:34.082500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:34.082774 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:34.083048 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:34.083049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:34.083050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:34.083128 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:34.083451 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:34.083460 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:34.083515 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 16:57:34.083598 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 16:57:34.083723 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":98,"threads_started":1}
I20260808 16:57:34.083943 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.075804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48978 (local address 127.8.14.126:40721)
0808 16:57:34.075856 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:34.075858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:34.075991 (+ 133us) server_negotiation.cc:407] Connection header received
0808 16:57:34.076060 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:34.076061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:34.076070 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:34.076123 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:34.076724 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.077218 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.078252 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.078431 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.078487 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.080944 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:34.080946 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:34.080946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:34.080952 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:34.082396 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:34.082868 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:34.082869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:34.082869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:34.082976 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:34.083236 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:34.083237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:34.083237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:34.083363 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:34.083434 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:34.083629 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 16:57:34.083778 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":28}
I20260808 16:57:34.086468 19429 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:48978:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 16:57:34.086702 19429 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:57:34.088214 19643 tablet_service.cc:1467] Processing CreateTablet for tablet d84054b8b53c468aac486c41c1fd87e3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e033336cc8714a12b360949bb22f37d3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:57:34.088235 19856 tablet_service.cc:1467] Processing CreateTablet for tablet d84054b8b53c468aac486c41c1fd87e3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e033336cc8714a12b360949bb22f37d3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:57:34.088423 19856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d84054b8b53c468aac486c41c1fd87e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:34.088424 19643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d84054b8b53c468aac486c41c1fd87e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:34.089744 20251 tablet_bootstrap.cc:492] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88: Bootstrap starting.
I20260808 16:57:34.090003 20254 tablet_bootstrap.cc:492] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce: Bootstrap starting.
I20260808 16:57:34.090103 20091 tablet_service.cc:1467] Processing CreateTablet for tablet d84054b8b53c468aac486c41c1fd87e3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e033336cc8714a12b360949bb22f37d3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 16:57:34.090246 20091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d84054b8b53c468aac486c41c1fd87e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:34.090246 20251 tablet_bootstrap.cc:654] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:34.090422 20254 tablet_bootstrap.cc:654] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:34.091121 20254 tablet_bootstrap.cc:492] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce: No bootstrap required, opened a new log
I20260808 16:57:34.091198 20254 ts_tablet_manager.cc:1397] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 16:57:34.091351 20254 raft_consensus.cc:348] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.091401 20254 raft_consensus.cc:374] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:34.091415 20254 raft_consensus.cc:729] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9f3e1c71eef49bca554d55fa2601dce, State: Initialized, Role: FOLLOWER
I20260808 16:57:34.091439 20254 consensus_queue.cc:260] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.091537 20254 ts_tablet_manager.cc:1428] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:34.091557 20260 tablet_replica.cc:439] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } }
I20260808 16:57:34.091645 20260 tablet_replica.cc:442] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:57:34.091820 20252 tablet_bootstrap.cc:492] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0: Bootstrap starting.
I20260808 16:57:34.092000 20251 tablet_bootstrap.cc:492] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88: No bootstrap required, opened a new log
I20260808 16:57:34.092056 20251 ts_tablet_manager.cc:1397] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:57:34.092188 20251 raft_consensus.cc:348] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.092243 20251 raft_consensus.cc:374] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:34.092258 20251 raft_consensus.cc:729] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77b30f4dc614497fb83c6db7b38fbe88, State: Initialized, Role: FOLLOWER
I20260808 16:57:34.092280 20252 tablet_bootstrap.cc:654] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:34.092327 20251 consensus_queue.cc:260] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.092429 20251 ts_tablet_manager.cc:1428] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:34.092495 20262 tablet_replica.cc:439] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } }
I20260808 16:57:34.092607 20262 tablet_replica.cc:442] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:57:34.093355 20252 tablet_bootstrap.cc:492] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0: No bootstrap required, opened a new log
I20260808 16:57:34.093412 20252 ts_tablet_manager.cc:1397] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:57:34.093583 20252 raft_consensus.cc:348] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.093643 20252 raft_consensus.cc:374] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:34.093658 20252 raft_consensus.cc:729] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0075aee99ed24683a938b7f6057b4cb0, State: Initialized, Role: FOLLOWER
I20260808 16:57:34.093693 20252 consensus_queue.cc:260] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.093807 20252 ts_tablet_manager.cc:1428] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:34.093870 20264 tablet_replica.cc:439] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } }
I20260808 16:57:34.093964 20264 tablet_replica.cc:442] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:57:34.210355 20264 raft_consensus.cc:482] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:57:34.210464 20264 raft_consensus.cc:504] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.210944 20264 leader_election.cc:288] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77b30f4dc614497fb83c6db7b38fbe88 (127.8.14.65:43729), c9f3e1c71eef49bca554d55fa2601dce (127.8.14.66:37889)
I20260808 16:57:34.213945 20249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.211344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:59797 (local address 127.8.14.65:43729)
0808 16:57:34.211426 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:34.211428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:34.211436 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:57:34.211470 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:34.211471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:34.211478 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:34.211542 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:34.211729 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.212228 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.213211 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.213575 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.213678 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.213689 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:34.213841 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 16:57:34.213878 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 16:57:34.214082 20030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.211100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:37889 (local address 127.8.14.67:46295)
0808 16:57:34.211175 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:34.211188 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:34.211231 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:34.211594 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:34.211595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:34.211597 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:34.211668 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:34.211675 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.212379 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.212380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:34.213169 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:34.213171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.213794 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.213795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:34.213882 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.213885 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:57:34.213961 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 16:57:34.214010 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 16:57:34.214082 20250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.211390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:46295 (local address 127.8.14.66:37889)
0808 16:57:34.211470 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:34.211472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:34.211480 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:57:34.211507 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:34.211508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:34.211515 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:34.211589 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:34.211759 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.212288 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.213273 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.213705 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.213795 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.213806 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:34.213969 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 16:57:34.214010 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 16:57:34.214114 20312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.211120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:43729 (local address 127.8.14.67:59797)
0808 16:57:34.211276 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:34.211289 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:34.211357 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:34.211546 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:34.211547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:34.211549 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:34.211647 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:34.211649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.212322 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.212323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:34.213110 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:34.213113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.213680 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.213681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:34.213759 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.213762 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:57:34.213833 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 16:57:34.213873 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":109,"threads_started":1}
I20260808 16:57:34.214375 19675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d84054b8b53c468aac486c41c1fd87e3" candidate_uuid: "0075aee99ed24683a938b7f6057b4cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77b30f4dc614497fb83c6db7b38fbe88" is_pre_election: true
I20260808 16:57:34.214474 19675 raft_consensus.cc:2454] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0075aee99ed24683a938b7f6057b4cb0 in term 0.
I20260808 16:57:34.214442 19910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d84054b8b53c468aac486c41c1fd87e3" candidate_uuid: "0075aee99ed24683a938b7f6057b4cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9f3e1c71eef49bca554d55fa2601dce" is_pre_election: true
I20260808 16:57:34.214636 19910 raft_consensus.cc:2454] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0075aee99ed24683a938b7f6057b4cb0 in term 0.
I20260808 16:57:34.214648 20025 leader_election.cc:302] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0075aee99ed24683a938b7f6057b4cb0, 77b30f4dc614497fb83c6db7b38fbe88; no voters:
I20260808 16:57:34.214804 20264 raft_consensus.cc:2781] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:57:34.214861 20264 raft_consensus.cc:482] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:57:34.214884 20264 raft_consensus.cc:3037] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:34.215785 20264 raft_consensus.cc:504] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.215956 20264 leader_election.cc:288] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [CANDIDATE]: Term 1 election: Requested vote from peers 77b30f4dc614497fb83c6db7b38fbe88 (127.8.14.65:43729), c9f3e1c71eef49bca554d55fa2601dce (127.8.14.66:37889)
I20260808 16:57:34.216284 19910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d84054b8b53c468aac486c41c1fd87e3" candidate_uuid: "0075aee99ed24683a938b7f6057b4cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9f3e1c71eef49bca554d55fa2601dce"
I20260808 16:57:34.216284 19675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d84054b8b53c468aac486c41c1fd87e3" candidate_uuid: "0075aee99ed24683a938b7f6057b4cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77b30f4dc614497fb83c6db7b38fbe88"
I20260808 16:57:34.216368 19910 raft_consensus.cc:3037] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:34.216368 19675 raft_consensus.cc:3037] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:34.217173 19910 raft_consensus.cc:2454] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0075aee99ed24683a938b7f6057b4cb0 in term 1.
I20260808 16:57:34.217173 19675 raft_consensus.cc:2454] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0075aee99ed24683a938b7f6057b4cb0 in term 1.
I20260808 16:57:34.217376 20025 leader_election.cc:302] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0075aee99ed24683a938b7f6057b4cb0, 77b30f4dc614497fb83c6db7b38fbe88; no voters:
I20260808 16:57:34.217487 20264 raft_consensus.cc:2781] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:57:34.217689 20264 raft_consensus.cc:686] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [term 1 LEADER]: Becoming Leader. State: Replica: 0075aee99ed24683a938b7f6057b4cb0, State: Running, Role: LEADER
I20260808 16:57:34.217779 20264 consensus_queue.cc:237] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.218156 20314 tablet_replica.cc:439] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0: TxnStatusTablet state changed. Reason: New leader 0075aee99ed24683a938b7f6057b4cb0. Latest consensus state: current_term: 1 leader_uuid: "0075aee99ed24683a938b7f6057b4cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } }
I20260808 16:57:34.218261 20314 tablet_replica.cc:442] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:57:34.218466 20317 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 16:57:34.218576 20317 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 16:57:34.218580 19428 catalog_manager.cc:5495] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0075aee99ed24683a938b7f6057b4cb0 (127.8.14.67). New cstate: current_term: 1 leader_uuid: "0075aee99ed24683a938b7f6057b4cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } health_report { overall_health: HEALTHY } } }
I20260808 16:57:34.240430 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.237431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42702 (local address 127.8.14.67:35341)
0808 16:57:34.237478 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:34.237480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:34.237517 (+ 37us) server_negotiation.cc:407] Connection header received
0808 16:57:34.237616 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:34.237617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:34.237632 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:34.237691 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:57:34.237871 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.238400 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.239644 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.239830 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.239884 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.240101 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:57:34.240209 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:57:34.240264 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:34.240353 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 16:57:34.240372 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:57:34.240494 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.237378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:35341 (local address 127.0.0.1:42702)
0808 16:57:34.237471 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:34.237482 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:34.237538 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:34.237723 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:34.237724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:34.237725 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:57:34.237780 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:34.237783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.238484 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.238486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:34.239561 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:34.239564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.239910 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.239911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:34.239983 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.240002 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:57:34.240279 (+ 277us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:57:34.240288 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 16:57:34.240349 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 16:57:34.240401 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 16:57:34.242589 19675 raft_consensus.cc:1261] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 FOLLOWER]: Refusing update from remote peer 0075aee99ed24683a938b7f6057b4cb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:57:34.242780 19910 raft_consensus.cc:1261] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce [term 1 FOLLOWER]: Refusing update from remote peer 0075aee99ed24683a938b7f6057b4cb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:57:34.242890 20264 consensus_queue.cc:1035] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:57:34.243247 20264 consensus_queue.cc:1035] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:57:34.243573 20262 tablet_replica.cc:439] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88: TxnStatusTablet state changed. Reason: New leader 0075aee99ed24683a938b7f6057b4cb0. Latest consensus state: current_term: 1 leader_uuid: "0075aee99ed24683a938b7f6057b4cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } }
I20260808 16:57:34.243681 20262 tablet_replica.cc:442] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:57:34.243882 20260 tablet_replica.cc:439] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce: TxnStatusTablet state changed. Reason: New leader 0075aee99ed24683a938b7f6057b4cb0. Latest consensus state: current_term: 1 leader_uuid: "0075aee99ed24683a938b7f6057b4cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } }
I20260808 16:57:34.243970 20260 tablet_replica.cc:442] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:57:34.244231 20264 tablet_replica.cc:439] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0075aee99ed24683a938b7f6057b4cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } }
I20260808 16:57:34.244309 20264 tablet_replica.cc:442] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:57:34.244635 20264 tablet_replica.cc:439] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0075aee99ed24683a938b7f6057b4cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } }
I20260808 16:57:34.244705 20264 tablet_replica.cc:442] T d84054b8b53c468aac486c41c1fd87e3 P 0075aee99ed24683a938b7f6057b4cb0: This TxnStatusTablet replica's current role is: LEADER
I20260808 16:57:34.245062 20275 mvcc.cc:205] Tried to move back new op lower bound from 7316309009375629312 to 7316309009276604416. Current Snapshot: MvccSnapshot[applied={T|T < 7316309009375629312}]
I20260808 16:57:34.245167 20277 mvcc.cc:205] Tried to move back new op lower bound from 7316309009375629312 to 7316309009276604416. Current Snapshot: MvccSnapshot[applied={T|T < 7316309009375629312}]
I20260808 16:57:34.245160 20260 tablet_replica.cc:439] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0075aee99ed24683a938b7f6057b4cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } }
I20260808 16:57:34.245226 20260 tablet_replica.cc:442] T d84054b8b53c468aac486c41c1fd87e3 P c9f3e1c71eef49bca554d55fa2601dce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:57:34.245260 20262 tablet_replica.cc:439] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0075aee99ed24683a938b7f6057b4cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } } }
I20260808 16:57:34.245328 20262 tablet_replica.cc:442] T d84054b8b53c468aac486c41c1fd87e3 P 77b30f4dc614497fb83c6db7b38fbe88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 16:57:34.251056 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.248719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:53163 (local address 127.8.14.67:35341)
0808 16:57:34.248756 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:34.248759 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:34.248902 (+ 143us) server_negotiation.cc:407] Connection header received
0808 16:57:34.248978 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:34.248979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:34.248990 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:34.249062 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:57:34.249185 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.249631 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.250383 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.250570 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.250645 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.250804 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:57:34.250842 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:57:34.250905 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:34.250972 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 16:57:34.250992 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 16:57:34.251096 20329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.248661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:35341 (local address 127.8.14.66:53163)
0808 16:57:34.248833 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:34.248843 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:34.248919 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:34.249065 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:34.249066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:34.249068 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:57:34.249116 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:34.249117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.249712 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.249714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:34.250293 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:34.250294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.250645 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.250646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:34.250703 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.250725 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:57:34.250921 (+ 196us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:57:34.250924 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:57:34.250971 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 16:57:34.251017 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260808 16:57:34.253155 20126 raft_consensus.cc:1261] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0 [term 1 FOLLOWER]: Refusing update from remote peer c9f3e1c71eef49bca554d55fa2601dce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:57:34.253273 19675 raft_consensus.cc:1261] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 FOLLOWER]: Refusing update from remote peer c9f3e1c71eef49bca554d55fa2601dce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:57:34.253369 20270 consensus_queue.cc:1035] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [LEADER]: Connected to new peer: Peer: permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:57:34.253615 20270 consensus_queue.cc:1035] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [LEADER]: Connected to new peer: Peer: permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:57:34.262216 20250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.259339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:38213 (local address 127.8.14.66:37889)
0808 16:57:34.259377 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:34.259380 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:34.259548 (+ 168us) server_negotiation.cc:407] Connection header received
0808 16:57:34.259621 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:34.259622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:34.259633 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:34.259685 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:57:34.259827 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.260406 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.261489 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.261684 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.261770 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.261967 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:57:34.262005 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:57:34.262067 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:34.262129 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 16:57:34.262149 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:57:34.262261 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.259295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:37889 (local address 127.8.14.67:38213)
0808 16:57:34.259452 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:34.259470 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:34.259563 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:34.259688 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:34.259688 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:34.259690 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:57:34.259750 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:34.259752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.260488 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.260489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:34.261409 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:34.261412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.261790 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.261791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:34.261857 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.261871 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:57:34.262069 (+ 198us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:57:34.262072 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:57:34.262124 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 16:57:34.262167 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260808 16:57:34.271028 19429 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:57:34.271147 19429 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:57:34.271828 19429 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8: Sending DeleteTablet for 3 replicas of tablet 0e1c73447834425d993798deeaed0956
I20260808 16:57:34.272158 19643 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1c73447834425d993798deeaed0956 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:34 UTC) from {username='oryx', principal='oryx/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:35258
I20260808 16:57:34.272162 20091 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1c73447834425d993798deeaed0956 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:34 UTC) from {username='oryx', principal='oryx/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:42692
I20260808 16:57:34.272279 20286 tablet_replica.cc:330] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88: stopping tablet replica
I20260808 16:57:34.272323 19856 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1c73447834425d993798deeaed0956 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:34 UTC) from {username='oryx', principal='oryx/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:38382
I20260808 16:57:34.272351 20286 raft_consensus.cc:2229] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:57:34.272353 20290 tablet_replica.cc:330] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0: stopping tablet replica
I20260808 16:57:34.272393 20286 raft_consensus.cc:2258] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:34.272400 20287 tablet_replica.cc:330] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce: stopping tablet replica
I20260808 16:57:34.272415 20290 raft_consensus.cc:2229] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:57:34.272450 20287 raft_consensus.cc:2229] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:57:34.272464 20290 raft_consensus.cc:2258] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:34.272513 20287 raft_consensus.cc:2258] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:34.272786 20290 ts_tablet_manager.cc:1905] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:34.272797 20286 ts_tablet_manager.cc:1905] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:34.272801 20287 ts_tablet_manager.cc:1905] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:34.274470 20286 ts_tablet_manager.cc:1918] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:57:34.274470 20287 ts_tablet_manager.cc:1918] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:57:34.274528 20287 log.cc:1196] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/0e1c73447834425d993798deeaed0956
I20260808 16:57:34.274528 20286 log.cc:1196] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/wal/wals/0e1c73447834425d993798deeaed0956
I20260808 16:57:34.274784 20286 ts_tablet_manager.cc:1939] T 0e1c73447834425d993798deeaed0956 P 77b30f4dc614497fb83c6db7b38fbe88: Deleting consensus metadata
I20260808 16:57:34.274799 20287 ts_tablet_manager.cc:1939] T 0e1c73447834425d993798deeaed0956 P c9f3e1c71eef49bca554d55fa2601dce: Deleting consensus metadata
I20260808 16:57:34.275147 19404 catalog_manager.cc:4841] TS 77b30f4dc614497fb83c6db7b38fbe88 (127.8.14.65:43729): tablet 0e1c73447834425d993798deeaed0956 (table test-table [id=127cfab454684266ba4510ca166c44dc]) successfully deleted
I20260808 16:57:34.275310 19400 catalog_manager.cc:4841] TS c9f3e1c71eef49bca554d55fa2601dce (127.8.14.66:37889): tablet 0e1c73447834425d993798deeaed0956 (table test-table [id=127cfab454684266ba4510ca166c44dc]) successfully deleted
I20260808 16:57:34.277436 20290 ts_tablet_manager.cc:1918] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:57:34.277519 20290 log.cc:1196] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/0e1c73447834425d993798deeaed0956
I20260808 16:57:34.277726 20290 ts_tablet_manager.cc:1939] T 0e1c73447834425d993798deeaed0956 P 0075aee99ed24683a938b7f6057b4cb0: Deleting consensus metadata
I20260808 16:57:34.278010 19402 catalog_manager.cc:4841] TS 0075aee99ed24683a938b7f6057b4cb0 (127.8.14.67:35341): tablet 0e1c73447834425d993798deeaed0956 (table test-table [id=127cfab454684266ba4510ca166c44dc]) successfully deleted
Aug 08 16:57:34 dist-test-slave-1nl6 krb5kdc[19357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 16:57:34 dist-test-slave-1nl6 krb5kdc[19357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:34 dist-test-slave-1nl6 krb5kdc[19357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 16:57:34.299973 19513 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:34.293098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48986 (local address 127.8.14.126:40721)
0808 16:57:34.293169 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:34.293171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:34.293355 (+ 184us) server_negotiation.cc:407] Connection header received
0808 16:57:34.293441 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:34.293442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:34.293468 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:34.293534 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:34.294376 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.295101 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.296264 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.296452 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.296507 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.299903 (+ 3396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48986: BlockingRecv error: recv got EOF from 127.0.0.1:48986 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 16:57:34 dist-test-slave-1nl6 krb5kdc[19357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM
I20260808 16:57:34.309033 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.301016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48990 (local address 127.8.14.126:40721)
0808 16:57:34.301081 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:34.301083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:34.301096 (+ 13us) server_negotiation.cc:407] Connection header received
0808 16:57:34.301178 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:34.301179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:34.301191 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:34.301266 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:34.301697 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.302504 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.303344 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.303662 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.303743 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.306176 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:34.306178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:34.306179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:34.306187 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:34.307874 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:34.308217 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:34.308219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:34.308219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:34.308290 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:34.308493 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:34.308494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:34.308495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:34.308657 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:34.308768 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:34.308850 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 16:57:34.308934 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 16:57:34.310624 19429 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48990:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:57:34.310784 19429 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:57:34.312372 19856 tablet_service.cc:1467] Processing CreateTablet for tablet 30801715e7c2430698308fe6aed62548 (DEFAULT_TABLE table=test-table [id=d9ccbeacfa9d40adadc36de5e3706723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:34.312376 19643 tablet_service.cc:1467] Processing CreateTablet for tablet 30801715e7c2430698308fe6aed62548 (DEFAULT_TABLE table=test-table [id=d9ccbeacfa9d40adadc36de5e3706723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:34.312525 19643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30801715e7c2430698308fe6aed62548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:34.312521 19856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30801715e7c2430698308fe6aed62548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:34.313483 20254 tablet_bootstrap.cc:492] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce: Bootstrap starting.
I20260808 16:57:34.313861 20254 tablet_bootstrap.cc:654] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:34.314038 20091 tablet_service.cc:1467] Processing CreateTablet for tablet 30801715e7c2430698308fe6aed62548 (DEFAULT_TABLE table=test-table [id=d9ccbeacfa9d40adadc36de5e3706723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:34.314178 20091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30801715e7c2430698308fe6aed62548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:34.314708 20254 tablet_bootstrap.cc:492] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce: No bootstrap required, opened a new log
I20260808 16:57:34.314745 20254 ts_tablet_manager.cc:1397] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 16:57:34.314857 20254 raft_consensus.cc:348] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.314970 20254 raft_consensus.cc:374] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:34.314999 20254 raft_consensus.cc:729] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9f3e1c71eef49bca554d55fa2601dce, State: Initialized, Role: FOLLOWER
I20260808 16:57:34.315056 20251 tablet_bootstrap.cc:492] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88: Bootstrap starting.
I20260808 16:57:34.315052 20254 consensus_queue.cc:260] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.315145 20254 ts_tablet_manager.cc:1428] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 16:57:34.315471 20251 tablet_bootstrap.cc:654] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:34.315611 20252 tablet_bootstrap.cc:492] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0: Bootstrap starting.
I20260808 16:57:34.316278 20251 tablet_bootstrap.cc:492] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88: No bootstrap required, opened a new log
I20260808 16:57:34.316340 20252 tablet_bootstrap.cc:654] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:34.316354 20251 ts_tablet_manager.cc:1397] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:57:34.316504 20251 raft_consensus.cc:348] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.316551 20251 raft_consensus.cc:374] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:34.316565 20251 raft_consensus.cc:729] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77b30f4dc614497fb83c6db7b38fbe88, State: Initialized, Role: FOLLOWER
I20260808 16:57:34.316619 20251 consensus_queue.cc:260] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.316711 20251 ts_tablet_manager.cc:1428] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:34.317471 20252 tablet_bootstrap.cc:492] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0: No bootstrap required, opened a new log
I20260808 16:57:34.317536 20252 ts_tablet_manager.cc:1397] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:57:34.317719 20252 raft_consensus.cc:348] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.317802 20252 raft_consensus.cc:374] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:34.317826 20252 raft_consensus.cc:729] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0075aee99ed24683a938b7f6057b4cb0, State: Initialized, Role: FOLLOWER
I20260808 16:57:34.317873 20252 consensus_queue.cc:260] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.317970 20252 ts_tablet_manager.cc:1428] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:34.662624 20262 raft_consensus.cc:482] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:57:34.662734 20262 raft_consensus.cc:504] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.663271 20262 leader_election.cc:288] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9f3e1c71eef49bca554d55fa2601dce (127.8.14.66:37889), 0075aee99ed24683a938b7f6057b4cb0 (127.8.14.67:35341)
I20260808 16:57:34.666435 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.663601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:55955 (local address 127.8.14.67:35341)
0808 16:57:34.663668 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:34.663671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:34.663790 (+ 119us) server_negotiation.cc:407] Connection header received
0808 16:57:34.663822 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:34.663823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:34.663833 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:34.663889 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:34.664063 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.664625 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.665685 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.666064 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.666119 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.666127 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:34.666322 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 16:57:34.666360 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 16:57:34.666455 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.663503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:35341 (local address 127.8.14.65:55955)
0808 16:57:34.663667 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:34.663678 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:34.663749 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:34.663911 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:34.663913 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:34.663915 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:34.663980 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:34.663982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.664739 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.664740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:34.665585 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:34.665587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.666172 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.666173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:34.666235 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.666239 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:57:34.666292 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 16:57:34.666350 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":33,"thread_start_us":72,"threads_started":1}
I20260808 16:57:34.666468 20250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.663570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:37217 (local address 127.8.14.66:37889)
0808 16:57:34.663646 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:34.663648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:34.663716 (+ 68us) server_negotiation.cc:407] Connection header received
0808 16:57:34.663836 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:34.663837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:34.663850 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:34.663925 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:34.664121 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.664627 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.665681 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.666080 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.666175 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.666185 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:34.666365 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 16:57:34.666408 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 16:57:34.666483 20376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.663492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:37889 (local address 127.8.14.65:37217)
0808 16:57:34.663647 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:34.663660 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:34.663736 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:34.663942 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:34.663943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:34.663946 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:34.664034 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:34.664037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.664734 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.664735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:34.665587 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:34.665589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.666197 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.666198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:34.666263 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.666267 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:57:34.666342 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 16:57:34.666379 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":2,"thread_start_us":104,"threads_started":1}
I20260808 16:57:34.666728 20126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30801715e7c2430698308fe6aed62548" candidate_uuid: "77b30f4dc614497fb83c6db7b38fbe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0075aee99ed24683a938b7f6057b4cb0" is_pre_election: true
I20260808 16:57:34.666832 20126 raft_consensus.cc:2454] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77b30f4dc614497fb83c6db7b38fbe88 in term 0.
I20260808 16:57:34.667042 19567 leader_election.cc:302] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0075aee99ed24683a938b7f6057b4cb0, 77b30f4dc614497fb83c6db7b38fbe88; no voters:
I20260808 16:57:34.667168 20262 raft_consensus.cc:2781] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:57:34.667145 19910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30801715e7c2430698308fe6aed62548" candidate_uuid: "77b30f4dc614497fb83c6db7b38fbe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9f3e1c71eef49bca554d55fa2601dce" is_pre_election: true
I20260808 16:57:34.667204 20262 raft_consensus.cc:482] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:57:34.667232 19910 raft_consensus.cc:2454] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77b30f4dc614497fb83c6db7b38fbe88 in term 0.
I20260808 16:57:34.667236 20262 raft_consensus.cc:3037] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:34.667981 20262 raft_consensus.cc:504] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.668136 20262 leader_election.cc:288] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [CANDIDATE]: Term 1 election: Requested vote from peers c9f3e1c71eef49bca554d55fa2601dce (127.8.14.66:37889), 0075aee99ed24683a938b7f6057b4cb0 (127.8.14.67:35341)
I20260808 16:57:34.668269 19910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30801715e7c2430698308fe6aed62548" candidate_uuid: "77b30f4dc614497fb83c6db7b38fbe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9f3e1c71eef49bca554d55fa2601dce"
I20260808 16:57:34.668357 19910 raft_consensus.cc:3037] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:34.668308 20126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30801715e7c2430698308fe6aed62548" candidate_uuid: "77b30f4dc614497fb83c6db7b38fbe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0075aee99ed24683a938b7f6057b4cb0"
I20260808 16:57:34.668385 20126 raft_consensus.cc:3037] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:34.669221 20126 raft_consensus.cc:2454] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77b30f4dc614497fb83c6db7b38fbe88 in term 1.
I20260808 16:57:34.669242 19910 raft_consensus.cc:2454] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77b30f4dc614497fb83c6db7b38fbe88 in term 1.
I20260808 16:57:34.669461 19565 leader_election.cc:302] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77b30f4dc614497fb83c6db7b38fbe88, c9f3e1c71eef49bca554d55fa2601dce; no voters:
I20260808 16:57:34.669571 20262 raft_consensus.cc:2781] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:57:34.669734 20262 raft_consensus.cc:686] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 LEADER]: Becoming Leader. State: Replica: 77b30f4dc614497fb83c6db7b38fbe88, State: Running, Role: LEADER
I20260808 16:57:34.669826 20262 consensus_queue.cc:237] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.670588 19429 catalog_manager.cc:5495] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 reported cstate change: term changed from 0 to 1, leader changed from <none> to 77b30f4dc614497fb83c6db7b38fbe88 (127.8.14.65). New cstate: current_term: 1 leader_uuid: "77b30f4dc614497fb83c6db7b38fbe88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } health_report { overall_health: UNKNOWN } } }
I20260808 16:57:34.760552 20384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:34.758022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35272 (local address 127.8.14.65:43729)
0808 16:57:34.758150 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:34.758152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:34.758162 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:57:34.758215 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:34.758215 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:34.758237 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:34.758316 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:57:34.758424 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.759034 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.759757 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:34.759984 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:34.760043 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:34.760163 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:57:34.760275 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:57:34.760332 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:34.760385 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 16:57:34.760402 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":68,"threads_started":1}
I20260808 16:57:34.761714 20126 raft_consensus.cc:1261] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0 [term 1 FOLLOWER]: Refusing update from remote peer 77b30f4dc614497fb83c6db7b38fbe88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:57:34.761906 19910 raft_consensus.cc:1261] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce [term 1 FOLLOWER]: Refusing update from remote peer 77b30f4dc614497fb83c6db7b38fbe88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:57:34.761996 20380 consensus_queue.cc:1035] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:57:34.762113 20262 consensus_queue.cc:1035] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:57:34.766048 19421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:57:34.766194 19421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:57:34.766866 19421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8: Sending DeleteTablet for 3 replicas of tablet 30801715e7c2430698308fe6aed62548
I20260808 16:57:34.767140 19856 tablet_service.cc:1514] Processing DeleteTablet for tablet 30801715e7c2430698308fe6aed62548 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:34 UTC) from {username='oryx', principal='oryx/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:38382
I20260808 16:57:34.767236 20287 tablet_replica.cc:330] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce: stopping tablet replica
I20260808 16:57:34.767243 19643 tablet_service.cc:1514] Processing DeleteTablet for tablet 30801715e7c2430698308fe6aed62548 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:34 UTC) from {username='oryx', principal='oryx/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:35258
I20260808 16:57:34.767297 20287 raft_consensus.cc:2229] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:57:34.767335 20286 tablet_replica.cc:330] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88: stopping tablet replica
I20260808 16:57:34.767349 20287 raft_consensus.cc:2258] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:34.767386 20286 raft_consensus.cc:2229] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:57:34.767465 20286 raft_consensus.cc:2258] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:34.767689 20091 tablet_service.cc:1514] Processing DeleteTablet for tablet 30801715e7c2430698308fe6aed62548 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:34 UTC) from {username='oryx', principal='oryx/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:42692
I20260808 16:57:34.767701 20287 ts_tablet_manager.cc:1905] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:34.767784 20290 tablet_replica.cc:330] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0: stopping tablet replica
I20260808 16:57:34.767832 20290 raft_consensus.cc:2229] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:57:34.767879 20290 raft_consensus.cc:2258] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:34.768182 20286 ts_tablet_manager.cc:1905] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:34.769126 20287 ts_tablet_manager.cc:1918] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:57:34.769192 20287 log.cc:1196] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/30801715e7c2430698308fe6aed62548
I20260808 16:57:34.769379 20287 ts_tablet_manager.cc:1939] T 30801715e7c2430698308fe6aed62548 P c9f3e1c71eef49bca554d55fa2601dce: Deleting consensus metadata
I20260808 16:57:34.769712 19400 catalog_manager.cc:4841] TS c9f3e1c71eef49bca554d55fa2601dce (127.8.14.66:37889): tablet 30801715e7c2430698308fe6aed62548 (table test-table [id=d9ccbeacfa9d40adadc36de5e3706723]) successfully deleted
I20260808 16:57:34.769774 20286 ts_tablet_manager.cc:1918] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:57:34.769831 20286 log.cc:1196] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-0/wal/wals/30801715e7c2430698308fe6aed62548
I20260808 16:57:34.770069 20286 ts_tablet_manager.cc:1939] T 30801715e7c2430698308fe6aed62548 P 77b30f4dc614497fb83c6db7b38fbe88: Deleting consensus metadata
I20260808 16:57:34.770421 19404 catalog_manager.cc:4841] TS 77b30f4dc614497fb83c6db7b38fbe88 (127.8.14.65:43729): tablet 30801715e7c2430698308fe6aed62548 (table test-table [id=d9ccbeacfa9d40adadc36de5e3706723]) successfully deleted
I20260808 16:57:34.770398 19421 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48990:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:57:34.770555 19421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:57:34.770941 20290 ts_tablet_manager.cc:1905] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:34.772161 20290 ts_tablet_manager.cc:1918] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:57:34.772217 20290 log.cc:1196] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/30801715e7c2430698308fe6aed62548
I20260808 16:57:34.772435 20290 ts_tablet_manager.cc:1939] T 30801715e7c2430698308fe6aed62548 P 0075aee99ed24683a938b7f6057b4cb0: Deleting consensus metadata
I20260808 16:57:34.772773 19402 catalog_manager.cc:4841] TS 0075aee99ed24683a938b7f6057b4cb0 (127.8.14.67:35341): tablet 30801715e7c2430698308fe6aed62548 (table test-table [id=d9ccbeacfa9d40adadc36de5e3706723]) successfully deleted
I20260808 16:57:34.773736 19643 tablet_service.cc:1467] Processing CreateTablet for tablet 84a6934af2054897b2d73d3a028bd7cc (DEFAULT_TABLE table=test-table [id=cbd9d9e01e8a46ed9a0aa0d659e70425]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:34.773759 19856 tablet_service.cc:1467] Processing CreateTablet for tablet 84a6934af2054897b2d73d3a028bd7cc (DEFAULT_TABLE table=test-table [id=cbd9d9e01e8a46ed9a0aa0d659e70425]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:34.773862 19643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84a6934af2054897b2d73d3a028bd7cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:34.773885 19856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84a6934af2054897b2d73d3a028bd7cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:34.774744 20254 tablet_bootstrap.cc:492] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce: Bootstrap starting.
I20260808 16:57:34.775060 20254 tablet_bootstrap.cc:654] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:34.775190 20251 tablet_bootstrap.cc:492] T 84a6934af2054897b2d73d3a028bd7cc P 77b30f4dc614497fb83c6db7b38fbe88: Bootstrap starting.
I20260808 16:57:34.775604 20251 tablet_bootstrap.cc:654] T 84a6934af2054897b2d73d3a028bd7cc P 77b30f4dc614497fb83c6db7b38fbe88: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:34.775657 20254 tablet_bootstrap.cc:492] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce: No bootstrap required, opened a new log
I20260808 16:57:34.775691 20254 ts_tablet_manager.cc:1397] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 16:57:34.775774 20254 raft_consensus.cc:348] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.775821 20254 raft_consensus.cc:374] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:34.775835 20254 raft_consensus.cc:729] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9f3e1c71eef49bca554d55fa2601dce, State: Initialized, Role: FOLLOWER
I20260808 16:57:34.775897 20254 consensus_queue.cc:260] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.775996 20254 ts_tablet_manager.cc:1428] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:34.776141 20091 tablet_service.cc:1467] Processing CreateTablet for tablet 84a6934af2054897b2d73d3a028bd7cc (DEFAULT_TABLE table=test-table [id=cbd9d9e01e8a46ed9a0aa0d659e70425]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:34.776273 20091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84a6934af2054897b2d73d3a028bd7cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:34.777554 20252 tablet_bootstrap.cc:492] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0: Bootstrap starting.
I20260808 16:57:34.777901 20252 tablet_bootstrap.cc:654] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:34.778060 20251 tablet_bootstrap.cc:492] T 84a6934af2054897b2d73d3a028bd7cc P 77b30f4dc614497fb83c6db7b38fbe88: No bootstrap required, opened a new log
I20260808 16:57:34.778105 20251 ts_tablet_manager.cc:1397] T 84a6934af2054897b2d73d3a028bd7cc P 77b30f4dc614497fb83c6db7b38fbe88: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 16:57:34.778259 20251 raft_consensus.cc:348] T 84a6934af2054897b2d73d3a028bd7cc P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.778332 20251 raft_consensus.cc:374] T 84a6934af2054897b2d73d3a028bd7cc P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:34.778355 20251 raft_consensus.cc:729] T 84a6934af2054897b2d73d3a028bd7cc P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77b30f4dc614497fb83c6db7b38fbe88, State: Initialized, Role: FOLLOWER
I20260808 16:57:34.778398 20251 consensus_queue.cc:260] T 84a6934af2054897b2d73d3a028bd7cc P 77b30f4dc614497fb83c6db7b38fbe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.778501 20251 ts_tablet_manager.cc:1428] T 84a6934af2054897b2d73d3a028bd7cc P 77b30f4dc614497fb83c6db7b38fbe88: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:34.778541 20252 tablet_bootstrap.cc:492] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0: No bootstrap required, opened a new log
I20260808 16:57:34.778586 20252 ts_tablet_manager.cc:1397] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 16:57:34.778708 20252 raft_consensus.cc:348] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.778792 20252 raft_consensus.cc:374] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:34.778831 20252 raft_consensus.cc:729] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0075aee99ed24683a938b7f6057b4cb0, State: Initialized, Role: FOLLOWER
I20260808 16:57:34.778870 20252 consensus_queue.cc:260] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:34.779121 20252 ts_tablet_manager.cc:1428] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:35.058851 20333 raft_consensus.cc:482] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:57:35.059008 20333 raft_consensus.cc:504] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:35.059219 20333 leader_election.cc:288] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77b30f4dc614497fb83c6db7b38fbe88 (127.8.14.65:43729), c9f3e1c71eef49bca554d55fa2601dce (127.8.14.66:37889)
I20260808 16:57:35.059425 19675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84a6934af2054897b2d73d3a028bd7cc" candidate_uuid: "0075aee99ed24683a938b7f6057b4cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77b30f4dc614497fb83c6db7b38fbe88" is_pre_election: true
I20260808 16:57:35.059532 19675 raft_consensus.cc:2454] T 84a6934af2054897b2d73d3a028bd7cc P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0075aee99ed24683a938b7f6057b4cb0 in term 0.
I20260808 16:57:35.059525 19910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84a6934af2054897b2d73d3a028bd7cc" candidate_uuid: "0075aee99ed24683a938b7f6057b4cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9f3e1c71eef49bca554d55fa2601dce" is_pre_election: true
I20260808 16:57:35.059687 19910 raft_consensus.cc:2454] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0075aee99ed24683a938b7f6057b4cb0 in term 0.
I20260808 16:57:35.059708 20025 leader_election.cc:302] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0075aee99ed24683a938b7f6057b4cb0, 77b30f4dc614497fb83c6db7b38fbe88; no voters:
I20260808 16:57:35.059805 20333 raft_consensus.cc:2781] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:57:35.059842 20333 raft_consensus.cc:482] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:57:35.059865 20333 raft_consensus.cc:3037] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:35.060639 20333 raft_consensus.cc:504] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:35.060792 20333 leader_election.cc:288] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [CANDIDATE]: Term 1 election: Requested vote from peers 77b30f4dc614497fb83c6db7b38fbe88 (127.8.14.65:43729), c9f3e1c71eef49bca554d55fa2601dce (127.8.14.66:37889)
I20260808 16:57:35.060942 19675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84a6934af2054897b2d73d3a028bd7cc" candidate_uuid: "0075aee99ed24683a938b7f6057b4cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77b30f4dc614497fb83c6db7b38fbe88"
I20260808 16:57:35.061028 19675 raft_consensus.cc:3037] T 84a6934af2054897b2d73d3a028bd7cc P 77b30f4dc614497fb83c6db7b38fbe88 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:35.061036 19910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84a6934af2054897b2d73d3a028bd7cc" candidate_uuid: "0075aee99ed24683a938b7f6057b4cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9f3e1c71eef49bca554d55fa2601dce"
I20260808 16:57:35.061121 19910 raft_consensus.cc:3037] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:35.061707 19675 raft_consensus.cc:2454] T 84a6934af2054897b2d73d3a028bd7cc P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0075aee99ed24683a938b7f6057b4cb0 in term 1.
I20260808 16:57:35.061730 19910 raft_consensus.cc:2454] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0075aee99ed24683a938b7f6057b4cb0 in term 1.
I20260808 16:57:35.061903 20020 leader_election.cc:302] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0075aee99ed24683a938b7f6057b4cb0, c9f3e1c71eef49bca554d55fa2601dce; no voters:
I20260808 16:57:35.062044 20333 raft_consensus.cc:2781] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:57:35.062202 20333 raft_consensus.cc:686] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [term 1 LEADER]: Becoming Leader. State: Replica: 0075aee99ed24683a938b7f6057b4cb0, State: Running, Role: LEADER
I20260808 16:57:35.062269 20333 consensus_queue.cc:237] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } }
I20260808 16:57:35.062933 19421 catalog_manager.cc:5495] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0075aee99ed24683a938b7f6057b4cb0 (127.8.14.67). New cstate: current_term: 1 leader_uuid: "0075aee99ed24683a938b7f6057b4cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0075aee99ed24683a938b7f6057b4cb0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35341 } health_report { overall_health: HEALTHY } } }
I20260808 16:57:35.133001 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:35.131066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42710 (local address 127.8.14.67:35341)
0808 16:57:35.131104 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:35.131106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:35.131115 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:57:35.131177 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:35.131178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:35.131186 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:35.131233 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:57:35.131324 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:35.131781 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:35.132462 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:35.132656 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:35.132716 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:35.132824 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:57:35.132849 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:57:35.132887 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:35.132921 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 16:57:35.132941 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:57:35.137125 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:35.135131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:41631 (local address 127.8.14.67:35341)
0808 16:57:35.135179 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:35.135181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:35.135316 (+ 135us) server_negotiation.cc:407] Connection header received
0808 16:57:35.135335 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:35.135335 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:35.135342 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:35.135383 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:57:35.135457 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:35.135847 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:35.136589 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:35.136749 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:35.136815 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:35.136947 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:57:35.136969 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:57:35.137000 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:35.137039 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 16:57:35.137054 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 16:57:35.137200 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:35.135068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:35341 (local address 127.8.14.67:41631)
0808 16:57:35.135229 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:35.135239 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:35.135295 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:35.135387 (+ 92us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:35.135387 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:35.135389 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 16:57:35.135424 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:35.135425 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:35.135904 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:35.135904 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:35.136514 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:35.136516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:35.136833 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:35.136834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:35.136872 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:35.136876 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 16:57:35.137008 (+ 132us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 16:57:35.137009 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 16:57:35.137037 (+ 28us) client_negotiation.cc:240] Negotiation successful
0808 16:57:35.137067 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":96,"threads_started":1}
I20260808 16:57:35.139880 19910 raft_consensus.cc:1261] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce [term 1 FOLLOWER]: Refusing update from remote peer 0075aee99ed24683a938b7f6057b4cb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:57:35.139891 19675 raft_consensus.cc:1261] T 84a6934af2054897b2d73d3a028bd7cc P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 FOLLOWER]: Refusing update from remote peer 0075aee99ed24683a938b7f6057b4cb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:57:35.140092 20333 consensus_queue.cc:1035] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9f3e1c71eef49bca554d55fa2601dce" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:57:35.140210 20403 consensus_queue.cc:1035] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77b30f4dc614497fb83c6db7b38fbe88" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:57:35.150759 19421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:57:35.150876 19421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:57:35.151518 19421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d0a853d2caa40f38e61c2e2fdb1f2e8: Sending DeleteTablet for 3 replicas of tablet 84a6934af2054897b2d73d3a028bd7cc
I20260808 16:57:35.151767 19856 tablet_service.cc:1514] Processing DeleteTablet for tablet 84a6934af2054897b2d73d3a028bd7cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:35 UTC) from {username='oryx', principal='oryx/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:38382
I20260808 16:57:35.151863 20287 tablet_replica.cc:330] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce: stopping tablet replica
I20260808 16:57:35.151844 19643 tablet_service.cc:1514] Processing DeleteTablet for tablet 84a6934af2054897b2d73d3a028bd7cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:35 UTC) from {username='oryx', principal='oryx/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:35258
I20260808 16:57:35.151909 20287 raft_consensus.cc:2229] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:57:35.151942 20287 raft_consensus.cc:2258] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:35.151935 20286 tablet_replica.cc:330] T 84a6934af2054897b2d73d3a028bd7cc P 77b30f4dc614497fb83c6db7b38fbe88: stopping tablet replica
I20260808 16:57:35.151981 20286 raft_consensus.cc:2229] T 84a6934af2054897b2d73d3a028bd7cc P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:57:35.152012 20286 raft_consensus.cc:2258] T 84a6934af2054897b2d73d3a028bd7cc P 77b30f4dc614497fb83c6db7b38fbe88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:35.152100 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 19537
I20260808 16:57:35.152299 20287 ts_tablet_manager.cc:1905] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:35.154603 20287 ts_tablet_manager.cc:1918] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:57:35.154659 20287 log.cc:1196] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/84a6934af2054897b2d73d3a028bd7cc
I20260808 16:57:35.154888 20287 ts_tablet_manager.cc:1939] T 84a6934af2054897b2d73d3a028bd7cc P c9f3e1c71eef49bca554d55fa2601dce: Deleting consensus metadata
I20260808 16:57:35.155223 19400 catalog_manager.cc:4841] TS c9f3e1c71eef49bca554d55fa2601dce (127.8.14.66:37889): tablet 84a6934af2054897b2d73d3a028bd7cc (table test-table [id=cbd9d9e01e8a46ed9a0aa0d659e70425]) successfully deleted
I20260808 16:57:35.160758 20091 tablet_service.cc:1514] Processing DeleteTablet for tablet 84a6934af2054897b2d73d3a028bd7cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:35 UTC) from {username='oryx', principal='oryx/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:42692
I20260808 16:57:35.160874 20290 tablet_replica.cc:330] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0: stopping tablet replica
I20260808 16:57:35.160936 20290 raft_consensus.cc:2229] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:57:35.161055 20290 raft_consensus.cc:2258] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:35.162305 20290 ts_tablet_manager.cc:1905] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:35.164458 20290 ts_tablet_manager.cc:1918] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 16:57:35.164534 20290 log.cc:1196] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/84a6934af2054897b2d73d3a028bd7cc
I20260808 16:57:35.164827 20290 ts_tablet_manager.cc:1939] T 84a6934af2054897b2d73d3a028bd7cc P 0075aee99ed24683a938b7f6057b4cb0: Deleting consensus metadata
I20260808 16:57:35.165424 19402 catalog_manager.cc:4841] TS 0075aee99ed24683a938b7f6057b4cb0 (127.8.14.67:35341): tablet 84a6934af2054897b2d73d3a028bd7cc (table test-table [id=cbd9d9e01e8a46ed9a0aa0d659e70425]) successfully deleted
W20260808 16:57:35.175568 19404 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.14.65:43729 (error 108)
I20260808 16:57:35.175791 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 19768
I20260808 16:57:35.181677 20417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:35.179838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:43729 (local address 127.0.0.1:35280)
0808 16:57:35.181505 (+ 1667us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:35.181559 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.65:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1645,"thread_start_us":46,"threads_started":1}
W20260808 16:57:35.181854 19404 catalog_manager.cc:4584] TS 77b30f4dc614497fb83c6db7b38fbe88 (127.8.14.65:43729): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 84a6934af2054897b2d73d3a028bd7cc: Network error: Client connection negotiation failed: client connection to 127.8.14.65:43729: connect: Connection refused (error 111)
I20260808 16:57:35.190644 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 19997
I20260808 16:57:35.207150 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 19378
2026-08-08T16:57:35Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3489 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:57:35 dist-test-slave-1nl6 krb5kdc[20421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:57:35 dist-test-slave-1nl6 krb5kdc[20421](info): set up 2 sockets
Aug 08 16:57:35 dist-test-slave-1nl6 krb5kdc[20421](info): commencing operation
krb5kdc: starting...
W20260808 16:57:36.429739 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:57:36 dist-test-slave-1nl6 krb5kdc[20421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:57:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:57:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:57:36.552099 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:40677
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:40677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:36.634989 20447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:36.635215 20447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:36.635253 20447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:36.636394 20447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:57:36.636463 20447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:36.636488 20447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:36.636508 20447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:57:36.636538 20447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:57:36.638036 20447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:40677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:40677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:36.638310 20447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:36.638533 20447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:36.643416 20456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:36.643415 20460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:36.643422 20457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:36.643482 20447 server_base.cc:1034] running on GCE node
I20260808 16:57:36.643922 20447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:36.644217 20447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:36.645381 20447 hybrid_clock.cc:648] HybridClock initialized: now 1786208256645365 us; error 26 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T16:57:36Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1434 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:57:36 dist-test-slave-1nl6 krb5kdc[20465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:57:36 dist-test-slave-1nl6 krb5kdc[20465](info): set up 2 sockets
Aug 08 16:57:36 dist-test-slave-1nl6 krb5kdc[20465](info): commencing operation
krb5kdc: starting...
W20260808 16:57:37.873835 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:57:37 dist-test-slave-1nl6 krb5kdc[20465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:57:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:57:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:57:37.997179 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:40981
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:40981
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:38.085600 20490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:38.085868 20490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:38.085891 20490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:38.087036 20490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:57:38.087080 20490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:38.087095 20490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:38.087108 20490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:57:38.087123 20490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:57:38.088529 20490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:40981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:40981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:38.088728 20490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:38.088891 20490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:57:38.093760 20490 server_base.cc:1034] running on GCE node
W20260808 16:57:38.093732 20503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:38.093734 20506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:38.093727 20498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:38.094180 20490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:38.094456 20490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:38.095600 20490 hybrid_clock.cc:648] HybridClock initialized: now 1786208258095586 us; error 29 us; skew 500 ppm
Aug 08 16:57:38 dist-test-slave-1nl6 krb5kdc[20465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208258, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:38.098068 20490 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:57:38.098580 20490 webserver.cc:460] Webserver started at http://127.8.14.126:38747/ using document root <none> and password file <none>
I20260808 16:57:38.098837 20490 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:38.098888 20490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:38.099112 20490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:38.100224 20490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "2976c5e43cdc43b9b14f39a719fa83d6"
format_stamp: "Formatted at 2026-08-08 16:57:38 on dist-test-slave-1nl6"
server_key: "867e1bff6d355ef58db27622a485b509"
server_key_iv: "5a0c4b3f8a2237ecfa642e9e026f10eb"
server_key_version: "encryptionkey@0"
I20260808 16:57:38.100636 20490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "2976c5e43cdc43b9b14f39a719fa83d6"
format_stamp: "Formatted at 2026-08-08 16:57:38 on dist-test-slave-1nl6"
server_key: "867e1bff6d355ef58db27622a485b509"
server_key_iv: "5a0c4b3f8a2237ecfa642e9e026f10eb"
server_key_version: "encryptionkey@0"
I20260808 16:57:38.102099 20490 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:57:38.102880 20520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:38.103111 20490 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:57:38.103222 20490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "2976c5e43cdc43b9b14f39a719fa83d6"
format_stamp: "Formatted at 2026-08-08 16:57:38 on dist-test-slave-1nl6"
server_key: "867e1bff6d355ef58db27622a485b509"
server_key_iv: "5a0c4b3f8a2237ecfa642e9e026f10eb"
server_key_version: "encryptionkey@0"
I20260808 16:57:38.103302 20490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:38.124423 20490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:38.126291 20490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:38.126442 20490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:38.130643 20490 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:40981
I20260808 16:57:38.130674 20607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:40981 every 8 connection(s)
I20260808 16:57:38.131103 20490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:57:38.131981 20609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:38.132771 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 20490
I20260808 16:57:38.132895 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:57:38.133113 8249 external_mini_cluster.cc:1428] Setting key ac5431d5471f74dfa7985c088eaf9f23
I20260808 16:57:38.135308 20609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6: Bootstrap starting.
I20260808 16:57:38.135941 20609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:38.136236 20609 log.cc:824] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6: Log is configured to *not* fsync() on all Append() calls
I20260808 16:57:38.137034 20609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6: No bootstrap required, opened a new log
I20260808 16:57:38.139844 20609 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2976c5e43cdc43b9b14f39a719fa83d6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40981 } }
I20260808 16:57:38.140026 20609 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:38.140072 20609 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2976c5e43cdc43b9b14f39a719fa83d6, State: Initialized, Role: FOLLOWER
I20260808 16:57:38.140244 20609 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2976c5e43cdc43b9b14f39a719fa83d6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40981 } }
I20260808 16:57:38.140355 20609 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:57:38.140419 20609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:57:38.140496 20609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 16:57:38 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:57:38.141201 20609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2976c5e43cdc43b9b14f39a719fa83d6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40981 } }
I20260808 16:57:38.141350 20609 leader_election.cc:302] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2976c5e43cdc43b9b14f39a719fa83d6; no voters:
I20260808 16:57:38.141515 20609 leader_election.cc:288] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:57:38.141773 20609 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:57:38.141780 20624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:57:38.141984 20624 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [term 1 LEADER]: Becoming Leader. State: Replica: 2976c5e43cdc43b9b14f39a719fa83d6, State: Running, Role: LEADER
I20260808 16:57:38.142084 20624 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2976c5e43cdc43b9b14f39a719fa83d6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40981 } }
I20260808 16:57:38.142405 20624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2976c5e43cdc43b9b14f39a719fa83d6. Latest consensus state: current_term: 1 leader_uuid: "2976c5e43cdc43b9b14f39a719fa83d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2976c5e43cdc43b9b14f39a719fa83d6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40981 } } }
I20260808 16:57:38.142472 20624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [sys.catalog]: This master's current role is: LEADER
I20260808 16:57:38.142458 20620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2976c5e43cdc43b9b14f39a719fa83d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2976c5e43cdc43b9b14f39a719fa83d6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40981 } } }
I20260808 16:57:38.142782 20620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [sys.catalog]: This master's current role is: LEADER
I20260808 16:57:38.143013 20636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:57:38.143626 20636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:57:38.146360 20636 catalog_manager.cc:1347] Generated new cluster ID: 1e6ac8c6fcb4479dab1068a58246add6
I20260808 16:57:38.146414 20636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:57:38.147161 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:38.133665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44960 (local address 127.8.14.126:40981)
0808 16:57:38.133846 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:38.133853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:38.133879 (+ 26us) server_negotiation.cc:407] Connection header received
0808 16:57:38.134306 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:38.134313 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:38.134358 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:38.134467 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:38.135328 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:38.136299 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:38.137317 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:38.137921 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:38.138002 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:38.141190 (+ 3188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:38.141195 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:38.141195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:38.141209 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:38.145432 (+ 4223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:38.145861 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:38.145864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:38.145865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:38.145972 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:38.146186 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:38.146187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:38.146187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:38.146481 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:38.146623 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:38.146778 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 16:57:38.146968 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260808 16:57:38.160818 20636 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:57:38.161307 20636 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:57:38.164448 20636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6: Generated new TSK 0
I20260808 16:57:38.164638 20636 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 16:57:38.191572 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:40981
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:38.272792 20659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:38.272985 20659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:38.273010 20659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:38.274129 20659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:38.274174 20659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:38.274199 20659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:57:38.275698 20659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:40981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:38.275931 20659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:38.276149 20659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:57:38.281105 20659 server_base.cc:1034] running on GCE node
W20260808 16:57:38.281155 20668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:38.281131 20670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:38.281229 20667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:38.281507 20659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:38.281767 20659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:38.282945 20659 hybrid_clock.cc:648] HybridClock initialized: now 1786208258282920 us; error 42 us; skew 500 ppm
Aug 08 16:57:38 dist-test-slave-1nl6 krb5kdc[20465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208258, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:38.285424 20659 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:57:38.285915 20659 webserver.cc:460] Webserver started at http://127.8.14.65:36895/ using document root <none> and password file <none>
I20260808 16:57:38.286145 20659 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:38.286190 20659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:38.286329 20659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:38.287364 20659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "368cdbbe1dd74a77a981e79aab41e6f6"
format_stamp: "Formatted at 2026-08-08 16:57:38 on dist-test-slave-1nl6"
server_key: "4c320e4870fef27438e850e7e92f2948"
server_key_iv: "2ca38ee2aced9474963d16a8a2a8ad83"
server_key_version: "encryptionkey@0"
I20260808 16:57:38.287719 20659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "368cdbbe1dd74a77a981e79aab41e6f6"
format_stamp: "Formatted at 2026-08-08 16:57:38 on dist-test-slave-1nl6"
server_key: "4c320e4870fef27438e850e7e92f2948"
server_key_iv: "2ca38ee2aced9474963d16a8a2a8ad83"
server_key_version: "encryptionkey@0"
I20260808 16:57:38.288998 20659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:57:38.289678 20680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:38.289846 20659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:57:38.289942 20659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "368cdbbe1dd74a77a981e79aab41e6f6"
format_stamp: "Formatted at 2026-08-08 16:57:38 on dist-test-slave-1nl6"
server_key: "4c320e4870fef27438e850e7e92f2948"
server_key_iv: "2ca38ee2aced9474963d16a8a2a8ad83"
server_key_version: "encryptionkey@0"
I20260808 16:57:38.290040 20659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:38.301671 20659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:38.303521 20659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:38.303686 20659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:38.303898 20659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:38.304211 20659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:38.304271 20659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:38.304328 20659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:38.304356 20659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:38.310262 20659 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:42709
I20260808 16:57:38.310313 20856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:42709 every 8 connection(s)
I20260808 16:57:38.310595 20659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:57:38.316143 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 20659
I20260808 16:57:38.316287 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
Aug 08 16:57:38 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208258, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:57:38.316509 8249 external_mini_cluster.cc:1428] Setting key 661824625ad4d85e12c27acdc3050362
I20260808 16:57:38.319967 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:38.311332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:60033 (local address 127.8.14.126:40981)
0808 16:57:38.311377 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:38.311379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:38.311560 (+ 181us) server_negotiation.cc:407] Connection header received
0808 16:57:38.312005 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:38.312006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:38.312016 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:38.312072 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:38.312955 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:38.313461 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:38.314410 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:38.314615 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:38.314672 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:38.316947 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:38.316949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:38.316950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:38.316958 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:38.318424 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:38.318991 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:38.318992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:38.318992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:38.319060 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:38.319352 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:38.319353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:38.319353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:38.319452 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:38.319503 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:38.319830 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 16:57:38.319903 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:57:38.320258 20860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:38.311278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:60033)
0808 16:57:38.311430 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:38.311468 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:38.311926 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:38.312135 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:38.312138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:38.312691 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:38.312870 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:38.312874 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:38.313550 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:38.313550 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:38.314321 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:38.314323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:38.314697 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:38.314697 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:38.314793 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:38.315238 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:38.315246 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:38.316838 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:38.318531 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:38.318539 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:38.318541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:38.318873 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:38.319132 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:38.319133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:38.319133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:38.319285 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:38.319517 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:38.319524 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:38.319682 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 16:57:38.319810 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 16:57:38.319989 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260808 16:57:38.320755 20857 heartbeater.cc:347] Connected to a master server at 127.8.14.126:40981
I20260808 16:57:38.320860 20857 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:38.321050 20857 heartbeater.cc:511] Master 127.8.14.126:40981 requested a full tablet report, sending...
I20260808 16:57:38.321543 20552 ts_manager.cc:194] Registered new tserver with Master: 368cdbbe1dd74a77a981e79aab41e6f6 (127.8.14.65:42709)
I20260808 16:57:38.322142 20552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:60033
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 16:57:38.341437 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:40981
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:38.423529 20864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:38.423723 20864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:38.423745 20864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:38.424782 20864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:38.424822 20864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:38.424847 20864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:57:38.426350 20864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:40981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:38.426529 20864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:38.426693 20864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:38.431555 20871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:38.431537 20872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:38.431533 20876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:38.431823 20864 server_base.cc:1034] running on GCE node
I20260808 16:57:38.432034 20864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:38.432300 20864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:38.433480 20864 hybrid_clock.cc:648] HybridClock initialized: now 1786208258433480 us; error 40 us; skew 500 ppm
Aug 08 16:57:38 dist-test-slave-1nl6 krb5kdc[20465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208258, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:38.436046 20864 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:57:38.436450 20864 webserver.cc:460] Webserver started at http://127.8.14.66:36869/ using document root <none> and password file <none>
I20260808 16:57:38.436625 20864 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:38.436663 20864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:38.436746 20864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:38.437702 20864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "4c1aae4c02b5491db65e8eaddf5a8f68"
format_stamp: "Formatted at 2026-08-08 16:57:38 on dist-test-slave-1nl6"
server_key: "6231db3183eb8676b5c6e08b80cce4d2"
server_key_iv: "13589ee6b69cced021e6121f5c8ea601"
server_key_version: "encryptionkey@0"
I20260808 16:57:38.438019 20864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "4c1aae4c02b5491db65e8eaddf5a8f68"
format_stamp: "Formatted at 2026-08-08 16:57:38 on dist-test-slave-1nl6"
server_key: "6231db3183eb8676b5c6e08b80cce4d2"
server_key_iv: "13589ee6b69cced021e6121f5c8ea601"
server_key_version: "encryptionkey@0"
I20260808 16:57:38.439340 20864 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:57:38.440021 20888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:38.440225 20864 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:57:38.440344 20864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "4c1aae4c02b5491db65e8eaddf5a8f68"
format_stamp: "Formatted at 2026-08-08 16:57:38 on dist-test-slave-1nl6"
server_key: "6231db3183eb8676b5c6e08b80cce4d2"
server_key_iv: "13589ee6b69cced021e6121f5c8ea601"
server_key_version: "encryptionkey@0"
I20260808 16:57:38.440424 20864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:38.465273 20864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:38.467222 20864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:38.467397 20864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:38.467643 20864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:38.467957 20864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:38.468012 20864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:38.468045 20864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:38.468073 20864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:38.473999 20864 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:36833
I20260808 16:57:38.474053 21062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:36833 every 8 connection(s)
I20260808 16:57:38.474408 20864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:57:38.476372 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 20864
I20260808 16:57:38.476485 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:57:38.476691 8249 external_mini_cluster.cc:1428] Setting key 481bf11ba9c1ac5c9feccaa1aae6cef8
Aug 08 16:57:38 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208258, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:57:38.484251 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:38.475153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:45321 (local address 127.8.14.126:40981)
0808 16:57:38.475196 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:38.475198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:38.475407 (+ 209us) server_negotiation.cc:407] Connection header received
0808 16:57:38.475862 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:38.475863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:38.475872 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:38.475941 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:38.477137 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:38.477805 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:38.478816 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:38.479101 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:38.479188 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:38.481328 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:38.481329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:38.481330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:38.481337 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:38.482788 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:38.483240 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:38.483241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:38.483242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:38.483312 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:38.483591 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:38.483592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:38.483592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:38.483706 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:38.483782 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:38.484060 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 16:57:38.484160 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:57:38.484441 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:38.475098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:45321)
0808 16:57:38.475275 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:38.475315 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:38.475780 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:38.476000 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:38.476003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:38.476792 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:38.477033 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:38.477042 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:38.477900 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:38.477902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:38.478708 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:38.478711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:38.479201 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:38.479202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:38.479340 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:38.479867 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:38.479883 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:38.481215 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:38.482923 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:38.482930 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:38.482932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:38.483157 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:38.483385 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:38.483386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:38.483386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:38.483512 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:38.483773 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:38.483780 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:38.483925 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 16:57:38.484035 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 16:57:38.484211 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260808 16:57:38.485067 21063 heartbeater.cc:347] Connected to a master server at 127.8.14.126:40981
I20260808 16:57:38.485178 21063 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:38.485356 21063 heartbeater.cc:511] Master 127.8.14.126:40981 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
I20260808 16:57:38.485816 20552 ts_manager.cc:194] Registered new tserver with Master: 4c1aae4c02b5491db65e8eaddf5a8f68 (127.8.14.66:36833)
I20260808 16:57:38.486227 20552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:45321
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 16:57:38.503247 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:40981
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:38.602660 21069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:38.602906 21069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:38.602989 21069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:38.604187 21069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:38.604229 21069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:38.604257 21069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:57:38.605806 21069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:40981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:38.606104 21069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:38.606385 21069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:38.612062 21083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:38.612138 21084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:38.612223 21069 server_base.cc:1034] running on GCE node
W20260808 16:57:38.612356 21089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:38.612591 21069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:38.612886 21069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:38.614050 21069 hybrid_clock.cc:648] HybridClock initialized: now 1786208258614033 us; error 33 us; skew 500 ppm
Aug 08 16:57:38 dist-test-slave-1nl6 krb5kdc[20465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208258, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:38.616549 21069 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:57:38.616976 21069 webserver.cc:460] Webserver started at http://127.8.14.67:38553/ using document root <none> and password file <none>
I20260808 16:57:38.617228 21069 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:38.617306 21069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:38.617457 21069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:38.618536 21069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "19bd94fc5f124226929695caa7aa8f68"
format_stamp: "Formatted at 2026-08-08 16:57:38 on dist-test-slave-1nl6"
server_key: "3c69a228d548b5c7f508113a86de5660"
server_key_iv: "e77dcb4e68a29694b8cd7d5cf758fcc6"
server_key_version: "encryptionkey@0"
I20260808 16:57:38.618949 21069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "19bd94fc5f124226929695caa7aa8f68"
format_stamp: "Formatted at 2026-08-08 16:57:38 on dist-test-slave-1nl6"
server_key: "3c69a228d548b5c7f508113a86de5660"
server_key_iv: "e77dcb4e68a29694b8cd7d5cf758fcc6"
server_key_version: "encryptionkey@0"
I20260808 16:57:38.620311 21069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:57:38.621045 21101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:38.621332 21069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:57:38.621438 21069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "19bd94fc5f124226929695caa7aa8f68"
format_stamp: "Formatted at 2026-08-08 16:57:38 on dist-test-slave-1nl6"
server_key: "3c69a228d548b5c7f508113a86de5660"
server_key_iv: "e77dcb4e68a29694b8cd7d5cf758fcc6"
server_key_version: "encryptionkey@0"
I20260808 16:57:38.621520 21069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:38.642649 21069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:38.645030 21069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:38.645198 21069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:38.645443 21069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:38.645797 21069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:38.645856 21069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:38.645905 21069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:38.645937 21069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:38.652022 21069 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:40501
I20260808 16:57:38.652065 21291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:40501 every 8 connection(s)
I20260808 16:57:38.652402 21069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
Aug 08 16:57:38 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208258, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:57:38.660653 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 21069
I20260808 16:57:38.660775 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:57:38.660960 8249 external_mini_cluster.cc:1428] Setting key 16438802ff629feddf223b10acf47c4a
I20260808 16:57:38.661566 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:38.653145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:42843 (local address 127.8.14.126:40981)
0808 16:57:38.653213 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:38.653215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:38.653409 (+ 194us) server_negotiation.cc:407] Connection header received
0808 16:57:38.653885 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:38.653886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:38.653896 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:38.653954 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:38.654841 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:38.655398 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:38.656342 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:38.656522 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:38.656573 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:38.658608 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:38.658610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:38.658610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:38.658617 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:38.660106 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:38.660482 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:38.660485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:38.660485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:38.660554 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:38.660799 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:38.660801 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:38.660801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:38.660947 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:38.661028 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:38.661375 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 16:57:38.661465 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 16:57:38.661790 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:38.653120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:42843)
0808 16:57:38.653274 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:38.653317 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:38.653804 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:38.654006 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:38.654009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:38.654562 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:38.654761 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:38.654766 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:38.655479 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:38.655480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:38.656269 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:38.656271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:38.656595 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:38.656596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:38.656693 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:38.657154 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:38.657163 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:38.658521 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:38.660175 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:38.660180 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:38.660182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:38.660395 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:38.660626 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:38.660626 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:38.660627 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:38.660719 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:38.661018 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:38.661026 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:38.661197 (+ 171us) client_negotiation.cc:769] Sending connection context
0808 16:57:38.661338 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 16:57:38.661524 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260808 16:57:38.662191 21296 heartbeater.cc:347] Connected to a master server at 127.8.14.126:40981
I20260808 16:57:38.662277 21296 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:38.662477 21296 heartbeater.cc:511] Master 127.8.14.126:40981 requested a full tablet report, sending...
I20260808 16:57:38.662860 20552 ts_manager.cc:194] Registered new tserver with Master: 19bd94fc5f124226929695caa7aa8f68 (127.8.14.67:40501)
I20260808 16:57:38.663269 20552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:42843
I20260808 16:57:38.672761 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.65.keytab.
Entry for principal oryx/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.65.keytab.
WARNING: no policy specified for oryx/127.8.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.66.keytab.
Entry for principal oryx/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.66.keytab.
WARNING: no policy specified for oryx/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.67.keytab.
Entry for principal oryx/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.67.keytab.
I20260808 16:57:38.743503 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 20659
I20260808 16:57:38.753203 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 20864
I20260808 16:57:38.764833 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 21069
I20260808 16:57:38.775141 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 20490
I20260808 16:57:38.786186 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:40981
--webserver_interface=127.8.14.126
--webserver_port=38747
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:40981
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:38.868698 21309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:38.868927 21309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:38.868965 21309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:38.870107 21309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:57:38.870188 21309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:38.870217 21309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:38.870257 21309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:57:38.870286 21309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:57:38.871763 21309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:40981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:40981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=38747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:38.872020 21309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:38.872265 21309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:38.877471 21321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:38.877646 21316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:38.877545 21309 server_base.cc:1034] running on GCE node
W20260808 16:57:38.877588 21317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:38.877975 21309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:38.878283 21309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:38.879458 21309 hybrid_clock.cc:648] HybridClock initialized: now 1786208258879426 us; error 48 us; skew 500 ppm
Aug 08 16:57:38 dist-test-slave-1nl6 krb5kdc[20465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208258, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:38.883145 21309 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:57:38.885342 21309 webserver.cc:460] Webserver started at http://127.8.14.126:38747/ using document root <none> and password file <none>
I20260808 16:57:38.885538 21309 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:38.885586 21309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:38.887491 21309 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:57:38.888192 21332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:38.888362 21309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:57:38.888464 21309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "2976c5e43cdc43b9b14f39a719fa83d6"
format_stamp: "Formatted at 2026-08-08 16:57:38 on dist-test-slave-1nl6"
server_key: "867e1bff6d355ef58db27622a485b509"
server_key_iv: "5a0c4b3f8a2237ecfa642e9e026f10eb"
server_key_version: "encryptionkey@0"
I20260808 16:57:38.888832 21309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:38.903422 21309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:38.905697 21309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:38.905869 21309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:38.910316 21309 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:40981
I20260808 16:57:38.910362 21416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:40981 every 8 connection(s)
I20260808 16:57:38.910790 21309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:57:38.911350 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 21309
I20260808 16:57:38.911676 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:42709
--local_ip_for_outbound_sockets=127.8.14.65
--tserver_master_addrs=127.8.14.126:40981
--webserver_port=36895
--webserver_interface=127.8.14.65
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.14.65
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:57:38.912510 21417 sys_catalog.cc:263] Verifying existing consensus state
I20260808 16:57:38.913195 21417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6: Bootstrap starting.
I20260808 16:57:38.923388 21417 log.cc:824] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6: Log is configured to *not* fsync() on all Append() calls
I20260808 16:57:38.925762 21417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:57:38.926137 21417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6: Bootstrap complete.
I20260808 16:57:38.929364 21417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2976c5e43cdc43b9b14f39a719fa83d6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40981 } }
I20260808 16:57:38.929682 21417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2976c5e43cdc43b9b14f39a719fa83d6, State: Initialized, Role: FOLLOWER
I20260808 16:57:38.929838 21417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2976c5e43cdc43b9b14f39a719fa83d6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40981 } }
I20260808 16:57:38.929903 21417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:57:38.929948 21417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:57:38.929994 21417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [term 1 FOLLOWER]: Advancing to term 2
I20260808 16:57:38.931293 21417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2976c5e43cdc43b9b14f39a719fa83d6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40981 } }
I20260808 16:57:38.931403 21417 leader_election.cc:302] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2976c5e43cdc43b9b14f39a719fa83d6; no voters:
I20260808 16:57:38.931565 21417 leader_election.cc:288] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 16:57:38.931650 21427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 16:57:38.931821 21417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:57:38.931833 21427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [term 2 LEADER]: Becoming Leader. State: Replica: 2976c5e43cdc43b9b14f39a719fa83d6, State: Running, Role: LEADER
I20260808 16:57:38.932054 21427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2976c5e43cdc43b9b14f39a719fa83d6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40981 } }
I20260808 16:57:38.932494 21429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2976c5e43cdc43b9b14f39a719fa83d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2976c5e43cdc43b9b14f39a719fa83d6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40981 } } }
I20260808 16:57:38.932669 21429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [sys.catalog]: This master's current role is: LEADER
I20260808 16:57:38.932505 21432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2976c5e43cdc43b9b14f39a719fa83d6. Latest consensus state: current_term: 2 leader_uuid: "2976c5e43cdc43b9b14f39a719fa83d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2976c5e43cdc43b9b14f39a719fa83d6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 40981 } } }
I20260808 16:57:38.932819 21432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6 [sys.catalog]: This master's current role is: LEADER
I20260808 16:57:38.933027 21448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:57:38.933602 21448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:57:38.933768 21448 catalog_manager.cc:1259] Loaded cluster ID: 1e6ac8c6fcb4479dab1068a58246add6
I20260808 16:57:38.933835 21448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:57:38.934639 21448 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:57:38.934849 21448 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2976c5e43cdc43b9b14f39a719fa83d6: Loaded TSK: 0
I20260808 16:57:38.935102 21448 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 16:57:39.019155 21424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:39.019366 21424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:39.019388 21424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:39.020561 21424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:39.020604 21424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:39.020629 21424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:57:39.022113 21424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.65.keytab
--principal=oryx/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:42709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=36895
--tserver_master_addrs=127.8.14.126:40981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:39.022439 21424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:39.022696 21424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:39.028014 21466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:39.028028 21471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:39.028051 21473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:39.028858 21424 server_base.cc:1034] running on GCE node
I20260808 16:57:39.029071 21424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:39.029354 21424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:39.030548 21424 hybrid_clock.cc:648] HybridClock initialized: now 1786208259030530 us; error 38 us; skew 500 ppm
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208259, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:39.033746 21424 init.cc:377] Logged in from keytab as oryx/127.8.14.65@KRBTEST.COM (short username oryx)
I20260808 16:57:39.034265 21424 webserver.cc:460] Webserver started at http://127.8.14.65:36895/ using document root <none> and password file <none>
I20260808 16:57:39.034518 21424 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:39.034605 21424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:39.036104 21424 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:57:39.036793 21484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:39.036976 21424 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:57:39.037086 21424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "368cdbbe1dd74a77a981e79aab41e6f6"
format_stamp: "Formatted at 2026-08-08 16:57:38 on dist-test-slave-1nl6"
server_key: "4c320e4870fef27438e850e7e92f2948"
server_key_iv: "2ca38ee2aced9474963d16a8a2a8ad83"
server_key_version: "encryptionkey@0"
I20260808 16:57:39.037452 21424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:39.053362 21424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:39.055639 21424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:39.055822 21424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:39.056118 21424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:39.056469 21424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:39.056525 21424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:39.056560 21424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:39.056586 21424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:39.064177 21424 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:42709
I20260808 16:57:39.064265 21673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:42709 every 8 connection(s)
I20260808 16:57:39.064607 21424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:57:39.069443 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 21424
I20260808 16:57:39.069792 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:36833
--local_ip_for_outbound_sockets=127.8.14.66
--tserver_master_addrs=127.8.14.126:40981
--webserver_port=36869
--webserver_interface=127.8.14.66
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.14.66
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:39.075330 21682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:39.065308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:60357)
0808 16:57:39.065618 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:39.065656 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:39.066099 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:39.066565 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:39.066568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:39.067221 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:39.067410 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.067416 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.069027 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.069028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.070262 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.070266 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.071228 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.071229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.071361 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.072047 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:39.072057 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:39.075069 (+ 3012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":219,"threads_started":1}
W20260808 16:57:39.075677 21674 heartbeater.cc:644] Failed to heartbeat to 127.8.14.126:40981 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.14.126:40981: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
W20260808 16:57:39.076442 21676 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:39.065416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:60357 (local address 127.8.14.126:40981)
0808 16:57:39.065734 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:39.065743 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:39.065782 (+ 39us) server_negotiation.cc:407] Connection header received
0808 16:57:39.066321 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:39.066328 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:39.066380 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:39.066518 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:39.067534 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.068918 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.070359 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.071005 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.071118 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.075242 (+ 4124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:60357: BlockingRecv error: recv got EOF from 127.8.14.65:60357 (error 108)
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":86,"threads_started":1}
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:39.081970 21682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:39.076048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:44821)
0808 16:57:39.076117 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:39.076130 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:39.076182 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:39.076488 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:39.076489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:39.076654 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:39.076717 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.076719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.077744 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.077745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.078583 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.078585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.079121 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.079122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.079197 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.079691 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:39.079696 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:39.081871 (+ 2175us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 16:57:39.082651 21686 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:39.076172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:44821 (local address 127.8.14.126:40981)
0808 16:57:39.076332 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:39.076334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:39.076343 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:57:39.076366 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:39.076367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:39.076381 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:39.076798 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:39.076838 (+ 40us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.077649 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.078677 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.079041 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.079121 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.081983 (+ 2862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:44821: BlockingRecv error: recv got EOF from 127.8.14.65:44821 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:39.087257 21682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:39.082313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:36417)
0808 16:57:39.082373 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:39.082385 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:39.082430 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:39.082601 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:39.082601 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:39.082740 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:39.082771 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.082772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.083731 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.083732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.084517 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.084519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.085002 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.085003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.085041 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.085504 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:39.085508 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:39.087215 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 16:57:39.087381 21676 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:39.082392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:36417 (local address 127.8.14.126:40981)
0808 16:57:39.082487 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:39.082489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:39.082497 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:57:39.082516 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:39.082517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:39.082527 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:39.082584 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:39.082856 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.083643 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.084615 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.084932 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.084999 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.087325 (+ 2326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:36417: BlockingRecv error: recv got EOF from 127.8.14.65:36417 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260808 16:57:39.087394 21674 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 16:57:39.155730 21684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:39.155941 21684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:39.155963 21684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:39.157245 21684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:39.157292 21684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:39.157321 21684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:57:39.158777 21684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.66.keytab
--principal=oryx/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:36833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=36869
--tserver_master_addrs=127.8.14.126:40981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:39.159075 21684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:39.159303 21684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:39.164418 21699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:39.164418 21694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:39.164405 21695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:39.164740 21684 server_base.cc:1034] running on GCE node
I20260808 16:57:39.164948 21684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:39.165210 21684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:39.166380 21684 hybrid_clock.cc:648] HybridClock initialized: now 1786208259166374 us; error 36 us; skew 500 ppm
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208259, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:39.168989 21684 init.cc:377] Logged in from keytab as oryx/127.8.14.66@KRBTEST.COM (short username oryx)
I20260808 16:57:39.169399 21684 webserver.cc:460] Webserver started at http://127.8.14.66:36869/ using document root <none> and password file <none>
I20260808 16:57:39.169651 21684 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:39.169741 21684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:39.171136 21684 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:57:39.171744 21709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:39.171923 21684 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:57:39.172030 21684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "4c1aae4c02b5491db65e8eaddf5a8f68"
format_stamp: "Formatted at 2026-08-08 16:57:38 on dist-test-slave-1nl6"
server_key: "6231db3183eb8676b5c6e08b80cce4d2"
server_key_iv: "13589ee6b69cced021e6121f5c8ea601"
server_key_version: "encryptionkey@0"
I20260808 16:57:39.172425 21684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:39.194242 21684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:39.196245 21684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:39.196413 21684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:39.196661 21684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:39.196979 21684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:39.197043 21684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:39.197108 21684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:39.197139 21684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:39.203034 21684 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:36833
I20260808 16:57:39.203130 21882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:36833 every 8 connection(s)
I20260808 16:57:39.203481 21684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:57:39.206431 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 21684
I20260808 16:57:39.206718 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:40501
--local_ip_for_outbound_sockets=127.8.14.67
--tserver_master_addrs=127.8.14.126:40981
--webserver_port=38553
--webserver_interface=127.8.14.67
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:38669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.14.67
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 16:57:39.211128 21676 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:39.204213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:49779 (local address 127.8.14.126:40981)
0808 16:57:39.204287 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:39.204289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:39.204513 (+ 224us) server_negotiation.cc:407] Connection header received
0808 16:57:39.205047 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:39.205049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:39.205063 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:39.205122 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:39.206079 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.206639 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.207735 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.207943 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.208027 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.211071 (+ 3044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:49779: BlockingRecv error: recv got EOF from 127.8.14.66:49779 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 16:57:39.211347 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:39.204215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:49779)
0808 16:57:39.204395 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:39.204431 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:39.204934 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:39.205206 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:39.205209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:39.205773 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:39.205954 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.205958 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.206743 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.206744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.207642 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.207644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.208039 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.208040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.208176 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.208656 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:39.208664 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:39.210989 (+ 2325us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":91,"threads_started":1}
W20260808 16:57:39.211580 21883 heartbeater.cc:644] Failed to heartbeat to 127.8.14.126:40981 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.14.126:40981: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:39.216159 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:39.211828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:56885)
0808 16:57:39.211880 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:39.211890 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:39.211953 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:39.212096 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:39.212098 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:39.212244 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:39.212282 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.212284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.212889 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.212890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.213374 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.213376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.213731 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.213732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.213797 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.214245 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:39.214249 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:39.216083 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 16:57:39.216238 21676 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:39.211901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:56885 (local address 127.8.14.126:40981)
0808 16:57:39.211957 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:39.211958 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:39.211967 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:57:39.212010 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:39.212012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:39.212023 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:39.212092 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:39.212362 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.212809 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.213457 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.213639 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.213730 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.216149 (+ 2419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:56885: BlockingRecv error: recv got EOF from 127.8.14.66:56885 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:39.221156 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:39.216614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:59971)
0808 16:57:39.216678 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:39.216689 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:39.216732 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:39.216888 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:39.216888 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:39.216993 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:39.217018 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.217018 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.217918 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.217919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.218365 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.218366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.218842 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.218843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.218888 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.219351 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:39.219355 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:39.221094 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 16:57:39.221269 21883 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 16:57:39.221449 21676 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:39.216688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:59971 (local address 127.8.14.126:40981)
0808 16:57:39.216755 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:39.216757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:39.216768 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:57:39.216794 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:39.216795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:39.216813 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:39.216880 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:39.217090 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.217841 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.218447 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.218747 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.218827 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.221365 (+ 2538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:59971: BlockingRecv error: recv got EOF from 127.8.14.66:59971 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 16:57:39.292352 21889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:39.292609 21889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:39.292651 21889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:39.293869 21889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:39.293934 21889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:39.293977 21889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:57:39.295403 21889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:38669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/krb5kdc/oryx_127.8.14.67.keytab
--principal=oryx/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:40501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=38553
--tserver_master_addrs=127.8.14.126:40981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:39.295660 21889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:39.295895 21889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:39.301000 21895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:39.301020 21896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:39.301031 21903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:39.301074 21889 server_base.cc:1034] running on GCE node
I20260808 16:57:39.301491 21889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:39.301692 21889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:39.302852 21889 hybrid_clock.cc:648] HybridClock initialized: now 1786208259302844 us; error 40 us; skew 500 ppm
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208259, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:39.305658 21889 init.cc:377] Logged in from keytab as oryx/127.8.14.67@KRBTEST.COM (short username oryx)
I20260808 16:57:39.306118 21889 webserver.cc:460] Webserver started at http://127.8.14.67:38553/ using document root <none> and password file <none>
I20260808 16:57:39.306342 21889 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:39.306422 21889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:39.307809 21889 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260808 16:57:39.308401 21913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:39.308574 21889 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:39.308673 21889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "19bd94fc5f124226929695caa7aa8f68"
format_stamp: "Formatted at 2026-08-08 16:57:38 on dist-test-slave-1nl6"
server_key: "3c69a228d548b5c7f508113a86de5660"
server_key_iv: "e77dcb4e68a29694b8cd7d5cf758fcc6"
server_key_version: "encryptionkey@0"
I20260808 16:57:39.309011 21889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:39.324030 21889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:39.325806 21889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:39.325966 21889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:39.326195 21889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:39.326499 21889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:39.326553 21889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:39.326594 21889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:39.326620 21889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:39.333415 21889 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:40501
I20260808 16:57:39.333467 22103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:40501 every 8 connection(s)
I20260808 16:57:39.333822 21889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:39.341948 22111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:39.334487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:53251)
0808 16:57:39.334669 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:39.334718 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:39.335293 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:39.335540 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:39.335544 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:39.336125 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:39.336308 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.336313 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.337032 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.337033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.338201 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.338204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.338773 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.338774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.338946 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.339557 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:39.339567 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:39.341811 (+ 2244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
W20260808 16:57:39.341948 21676 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:39.334527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:53251 (local address 127.8.14.126:40981)
0808 16:57:39.334574 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:39.334576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:39.334815 (+ 239us) server_negotiation.cc:407] Connection header received
0808 16:57:39.335402 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:39.335403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:39.335416 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:39.335475 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:39.336423 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.336921 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.338311 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.338665 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.338783 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.341902 (+ 3119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.67:53251: BlockingRecv error: recv got EOF from 127.8.14.67:53251 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 16:57:39.342170 22106 heartbeater.cc:644] Failed to heartbeat to 127.8.14.126:40981 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.14.126:40981: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
I20260808 16:57:39.342840 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 21889
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:39.347209 22111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:39.342404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:53489)
0808 16:57:39.342453 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:39.342463 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:39.342500 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:39.342757 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:39.342758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:39.342904 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:39.342984 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.342985 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.343859 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.343860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.344452 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.344455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.344785 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.344786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.344849 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.345322 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:39.345327 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:39.347124 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 16:57:39.347275 21676 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:39.342571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:53489 (local address 127.8.14.126:40981)
0808 16:57:39.342639 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:39.342641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:39.342648 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:57:39.342681 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:39.342682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:39.342690 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:39.342752 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:39.343057 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.343778 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.344534 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.344713 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.344764 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.347210 (+ 2446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.67:53489: BlockingRecv error: recv got EOF from 127.8.14.67:53489 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 16:57:39.351119 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:39.343292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44964 (local address 127.8.14.126:40981)
0808 16:57:39.343341 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:39.343343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:39.343499 (+ 156us) server_negotiation.cc:407] Connection header received
0808 16:57:39.343558 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:39.343559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:39.343568 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:39.343610 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:39.344241 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.344846 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.345748 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.345947 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.345995 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.347699 (+ 1704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:39.347704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:39.347704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:39.347716 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:39.349716 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:39.350055 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:39.350058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:39.350059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:39.350165 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:39.350371 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:39.350372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:39.350373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:39.350596 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:39.350743 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:39.350857 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 16:57:39.351022 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:39 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:39.352368 22111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:39.347439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:37081)
0808 16:57:39.347487 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:39.347498 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:39.347546 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:39.347988 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:39.347989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:39.348084 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:39.348110 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.348111 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.348872 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.348873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.349628 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:39.349631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.350048 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.350049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:39.350107 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.350505 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:39.350510 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:39.352329 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 16:57:39.352487 22106 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 16:57:39.352509 21676 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:39.347472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:37081 (local address 127.8.14.126:40981)
0808 16:57:39.347499 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:39.347500 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:39.347533 (+ 33us) server_negotiation.cc:407] Connection header received
0808 16:57:39.347605 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:39.347606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:39.347612 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:39.347661 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:39.348198 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.348785 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.349740 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:39.349928 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:39.350049 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:39.352440 (+ 2391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.67:37081: BlockingRecv error: recv got EOF from 127.8.14.67:37081 (error 108)
Metrics: {"server-negotiator.queue_time_us":13}
Aug 08 16:57:40 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:40 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:40.092510 22115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:40.087729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:49171)
0808 16:57:40.087888 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:40.087898 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:40.087959 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:40.088086 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:40.088086 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:40.088250 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:40.088299 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:40.088300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:40.088934 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:40.088935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:40.089471 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:40.089473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:40.089887 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:40.089889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:40.089964 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:40.090433 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:40.090437 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:40.092402 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":102,"threads_started":1}
W20260808 16:57:40.092557 22117 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:40.087814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:49171 (local address 127.8.14.126:40981)
0808 16:57:40.087933 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:40.087935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:40.087944 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:57:40.088015 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:40.088016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:40.088025 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:40.088081 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:40.088354 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:40.088858 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:40.089543 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:40.089731 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:40.089792 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:40.092483 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:49171: BlockingRecv error: recv got EOF from 127.8.14.65:49171 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":55,"threads_started":1}
Aug 08 16:57:40 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:40 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:40.226516 22118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:40.221730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:56841)
0808 16:57:40.221869 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:40.221879 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:40.221960 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:40.222112 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:40.222113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:40.222283 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:40.222331 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:40.222333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:40.223086 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:40.223087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:40.223667 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:40.223669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:40.224032 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:40.224033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:40.224088 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:40.224558 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:40.224563 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:40.226419 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":56,"threads_started":1}
W20260808 16:57:40.226612 22117 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:40.221812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:56841 (local address 127.8.14.126:40981)
0808 16:57:40.221904 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:40.221906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:40.221944 (+ 38us) server_negotiation.cc:407] Connection header received
0808 16:57:40.222019 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:40.222020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:40.222030 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:40.222092 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:40.222413 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:40.222957 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:40.223750 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:40.223937 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:40.223989 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:40.226542 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:56841: BlockingRecv error: recv got EOF from 127.8.14.66:56841 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 16:57:40 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:40 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:40.357795 22119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:40.352897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:59703)
0808 16:57:40.353030 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:40.353049 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:40.353104 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:40.353274 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:40.353274 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:40.353452 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:40.353502 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:40.353504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:40.354122 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:40.354123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:40.354658 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:40.354660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:40.355054 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:40.355056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:40.355124 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:40.355784 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:40.355791 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:40.357722 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
W20260808 16:57:40.357898 22117 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:40.352991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:59703 (local address 127.8.14.126:40981)
0808 16:57:40.353064 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:40.353066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:40.353089 (+ 23us) server_negotiation.cc:407] Connection header received
0808 16:57:40.353181 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:40.353182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:40.353194 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:40.353252 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:40.353568 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:40.354049 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:40.354740 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:40.354967 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:40.355021 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:40.357808 (+ 2787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.67:59703: BlockingRecv error: recv got EOF from 127.8.14.67:59703 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 08 16:57:41 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:41 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:41.097733 22120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:41.092972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:59551)
0808 16:57:41.093137 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:41.093147 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:41.093200 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:41.093348 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:41.093349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:41.093516 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:41.093566 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:41.093568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:41.094200 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:41.094201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:41.094716 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:41.094717 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:41.095179 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:41.095180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:41.095224 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:41.095716 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:41.095720 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:41.097669 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":101,"threads_started":1}
W20260808 16:57:41.097879 22122 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:41.093065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:59551 (local address 127.8.14.126:40981)
0808 16:57:41.093181 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:41.093182 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:41.093191 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:57:41.093262 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:41.093263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:41.093272 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:41.093329 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:41.093633 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:41.094134 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:41.094812 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:41.095096 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:41.095159 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:41.097781 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:59551: BlockingRecv error: recv got EOF from 127.8.14.65:59551 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
Aug 08 16:57:41 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:41 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:41.231757 22123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:41.227064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:42075)
0808 16:57:41.227203 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:41.227215 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:41.227281 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:41.227408 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:41.227409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:41.227570 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:41.227629 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:41.227630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:41.228240 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:41.228241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:41.228757 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:41.228759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:41.229064 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:41.229064 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:41.229102 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:41.229529 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:41.229534 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:41.231647 (+ 2113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
W20260808 16:57:41.231805 22122 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:41.227162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:42075 (local address 127.8.14.126:40981)
0808 16:57:41.227217 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:41.227219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:41.227264 (+ 45us) server_negotiation.cc:407] Connection header received
0808 16:57:41.227335 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:41.227336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:41.227347 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:41.227390 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:41.227693 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:41.228177 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:41.228824 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:41.229004 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:41.229053 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:41.231752 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:42075: BlockingRecv error: recv got EOF from 127.8.14.66:42075 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 16:57:41 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:41 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:41.363049 22126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:41.358248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:36747)
0808 16:57:41.358395 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:41.358410 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:41.358486 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:41.358669 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:41.358670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:41.358862 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:41.358924 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:41.358926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:41.359702 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:41.359703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:41.360292 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:41.360294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:41.360656 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:41.360657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:41.360715 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:41.361172 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:41.361176 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:41.362954 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":60,"threads_started":1}
W20260808 16:57:41.363092 22122 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:41.358381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:36747 (local address 127.8.14.126:40981)
0808 16:57:41.358430 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:41.358432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:41.358464 (+ 32us) server_negotiation.cc:407] Connection header received
0808 16:57:41.358551 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:41.358553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:41.358568 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:41.358643 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:41.359014 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:41.359612 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:41.360390 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:41.360580 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:41.360658 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:41.363028 (+ 2370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.67:36747: BlockingRecv error: recv got EOF from 127.8.14.67:36747 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 16:57:42 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:42 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:42.102691 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:42.098244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:35677)
0808 16:57:42.098400 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:42.098417 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:42.098483 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:42.098631 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:42.098632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:42.098806 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:42.098854 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:42.098856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:42.099537 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:42.099538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:42.100070 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:42.100071 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:42.100409 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:42.100410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:42.100449 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:42.100868 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:42.100873 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:42.102633 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
W20260808 16:57:42.102854 22129 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:42.098324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:35677 (local address 127.8.14.126:40981)
0808 16:57:42.098506 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:42.098508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:42.098516 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:57:42.098538 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:42.098538 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:42.098548 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:42.098611 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:42.098969 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:42.099468 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:42.100139 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:42.100336 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:42.100396 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:42.102754 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:35677: BlockingRecv error: recv got EOF from 127.8.14.65:35677 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
Aug 08 16:57:42 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:42 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:42.237066 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:42.232235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:34399)
0808 16:57:42.232362 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:42.232374 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:42.232439 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:42.232571 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:42.232571 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:42.232732 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:42.232826 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:42.232829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:42.233633 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:42.233635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:42.234178 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:42.234180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:42.234541 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:42.234542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:42.234583 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:42.235094 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:42.235099 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:42.236940 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
W20260808 16:57:42.237102 22129 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:42.232303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:34399 (local address 127.8.14.126:40981)
0808 16:57:42.232353 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:42.232355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:42.232437 (+ 82us) server_negotiation.cc:407] Connection header received
0808 16:57:42.232486 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:42.232487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:42.232498 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:42.232554 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:42.232896 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:42.233562 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:42.234277 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:42.234467 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:42.234529 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:42.237036 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:34399: BlockingRecv error: recv got EOF from 127.8.14.66:34399 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 08 16:57:42 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:42 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:42.368542 22131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:42.363534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:49245)
0808 16:57:42.363774 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:42.363785 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:42.363884 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:42.364044 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:42.364046 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:42.364239 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:42.364285 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:42.364287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:42.364958 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:42.364960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:42.365548 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:42.365549 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:42.365941 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:42.365942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:42.366000 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:42.366452 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:42.366457 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:42.368472 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":153,"threads_started":1}
W20260808 16:57:42.368656 22129 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:42.363681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:49245 (local address 127.8.14.126:40981)
0808 16:57:42.363767 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:42.363769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:42.363864 (+ 95us) server_negotiation.cc:407] Connection header received
0808 16:57:42.363941 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:42.363942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:42.363959 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:42.364036 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:42.364381 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:42.364876 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:42.365654 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:42.365858 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:42.365943 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:42.368559 (+ 2616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.67:49245: BlockingRecv error: recv got EOF from 127.8.14.67:49245 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 16:57:43 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:43 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:43.107678 22132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:43.103201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:55859)
0808 16:57:43.103333 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:43.103349 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:43.103398 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:43.103526 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:43.103527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:43.103688 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:43.103737 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:43.103739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:43.104361 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:43.104362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:43.104875 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:43.104876 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:43.105248 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:43.105248 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:43.105287 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:43.105735 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:43.105740 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:43.107619 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
W20260808 16:57:43.107834 22133 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:43.103258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:55859 (local address 127.8.14.126:40981)
0808 16:57:43.103390 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:43.103392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:43.103402 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:57:43.103458 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:43.103459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:43.103470 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:43.103522 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:43.103797 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:43.104291 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:43.104961 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:43.105177 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:43.105238 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:43.107737 (+ 2499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:55859: BlockingRecv error: recv got EOF from 127.8.14.65:55859 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
Aug 08 16:57:43 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:43 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:43.242379 22135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:43.237610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:58247)
0808 16:57:43.237759 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:43.237772 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:43.237860 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:43.238036 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:43.238038 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:43.238279 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:43.238334 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:43.238336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:43.238968 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:43.238969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:43.239556 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:43.239558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:43.239870 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:43.239871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:43.239910 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:43.240420 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:43.240425 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:43.242282 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
W20260808 16:57:43.242429 22133 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:43.237707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:58247 (local address 127.8.14.126:40981)
0808 16:57:43.237762 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:43.237765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:43.237886 (+ 121us) server_negotiation.cc:407] Connection header received
0808 16:57:43.237916 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:43.237917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:43.237930 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:43.237986 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:43.238403 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:43.238892 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:43.239625 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:43.239810 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:43.239856 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:43.242364 (+ 2508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:58247: BlockingRecv error: recv got EOF from 127.8.14.66:58247 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 08 16:57:43 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:43 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:43.373864 22136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:43.368995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:35263)
0808 16:57:43.369213 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:43.369227 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:43.369329 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:43.369502 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:43.369503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:43.369710 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:43.369758 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:43.369759 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:43.370455 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:43.370456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:43.371021 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:43.371023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:43.371450 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:43.371451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:43.371510 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:43.371959 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:43.371963 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:43.373809 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":97,"threads_started":1}
W20260808 16:57:43.373986 22133 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:43.369105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:35263 (local address 127.8.14.126:40981)
0808 16:57:43.369212 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:43.369215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:43.369310 (+ 95us) server_negotiation.cc:407] Connection header received
0808 16:57:43.369401 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:43.369403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:43.369418 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:43.369495 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:43.369848 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:43.370357 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:43.371125 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:43.371352 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:43.371452 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:43.373905 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.67:35263: BlockingRecv error: recv got EOF from 127.8.14.67:35263 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 08 16:57:44 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:44 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:44.116252 22137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:44.108281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:37683)
0808 16:57:44.108552 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:44.108564 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:44.108622 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:44.108766 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:44.108767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:44.108940 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:44.108990 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:44.108992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:44.109619 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:44.109620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:44.110141 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:44.110143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:44.110462 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:44.110462 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:44.110502 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:44.114312 (+ 3810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:44.114317 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:44.116182 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":191,"threads_started":1}
W20260808 16:57:44.116407 22139 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:44.108481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:37683 (local address 127.8.14.126:40981)
0808 16:57:44.108636 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:44.108638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:44.108652 (+ 14us) server_negotiation.cc:407] Connection header received
0808 16:57:44.108674 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:44.108675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:44.108684 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:44.108745 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:44.109057 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:44.109555 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:44.110210 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:44.110398 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:44.110448 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:44.116310 (+ 5862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:37683: BlockingRecv error: recv got EOF from 127.8.14.65:37683 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":102,"threads_started":1}
Aug 08 16:57:44 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:44 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:44.247537 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:44.242978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:53855)
0808 16:57:44.243141 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:44.243154 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:44.243208 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:44.243396 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:44.243397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:44.243580 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:44.243630 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:44.243631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:44.244324 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:44.244325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:44.244832 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:44.244833 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:44.245185 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:44.245186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:44.245225 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:44.245668 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:44.245673 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:44.247478 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
W20260808 16:57:44.247646 22139 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:44.243098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:53855 (local address 127.8.14.126:40981)
0808 16:57:44.243171 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:44.243173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:44.243203 (+ 30us) server_negotiation.cc:407] Connection header received
0808 16:57:44.243288 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:44.243289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:44.243299 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:44.243371 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:44.243700 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:44.244249 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:44.244904 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:44.245089 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:44.245168 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:44.247570 (+ 2402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:53855: BlockingRecv error: recv got EOF from 127.8.14.66:53855 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 16:57:44 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:44 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:44.379344 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:44.374368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:48321)
0808 16:57:44.374513 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:44.374524 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:44.374620 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:44.374787 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:44.374788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:44.375045 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:44.375093 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:44.375095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:44.375853 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:44.375854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:44.376388 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:44.376390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:44.376791 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:44.376792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:44.376836 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:44.377357 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:44.377361 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:44.379274 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
W20260808 16:57:44.379477 22139 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:44.374447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:48321 (local address 127.8.14.126:40981)
0808 16:57:44.374510 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:44.374513 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:44.374598 (+ 85us) server_negotiation.cc:407] Connection header received
0808 16:57:44.374683 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:44.374684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:44.374700 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:44.374779 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:44.375183 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:44.375770 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:44.376524 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:44.376724 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:44.376779 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:44.379373 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.67:48321: BlockingRecv error: recv got EOF from 127.8.14.67:48321 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 08 16:57:45 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:45 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:45.121562 22143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:45.116778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:56511)
0808 16:57:45.116920 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:45.116931 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:45.117007 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:45.117325 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:45.117327 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:45.117542 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:45.117595 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:45.117597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:45.118252 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:45.118252 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:45.118793 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:45.118794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:45.119198 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:45.119198 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:45.119242 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:45.119700 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:45.119704 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:45.121500 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
W20260808 16:57:45.121675 22144 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:45.116857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:56511 (local address 127.8.14.126:40981)
0808 16:57:45.116958 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:45.116960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:45.116970 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:57:45.117172 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:45.117174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:45.117217 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:45.117307 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:45.117661 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:45.118182 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:45.118877 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:45.119106 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:45.119188 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:45.121596 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:56511: BlockingRecv error: recv got EOF from 127.8.14.65:56511 (error 108)
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":52,"threads_started":1}
Aug 08 16:57:45 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:45 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:45.252527 22146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:45.248036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:51199)
0808 16:57:45.248199 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:45.248210 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:45.248260 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:45.248400 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:45.248401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:45.248575 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:45.248623 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:45.248624 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:45.249252 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:45.249253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:45.249764 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:45.249765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:45.250072 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:45.250073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:45.250114 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:45.250543 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:45.250547 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:45.252432 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":115,"threads_started":1}
W20260808 16:57:45.252579 22144 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:45.248118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:51199 (local address 127.8.14.126:40981)
0808 16:57:45.248232 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:45.248235 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:45.248247 (+ 12us) server_negotiation.cc:407] Connection header received
0808 16:57:45.248320 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:45.248320 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:45.248330 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:45.248382 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:45.248683 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:45.249187 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:45.249832 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:45.250010 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:45.250059 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:45.252513 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:51199: BlockingRecv error: recv got EOF from 127.8.14.66:51199 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 08 16:57:45 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:45 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:45.384411 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:45.379812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:50137)
0808 16:57:45.379971 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:45.379982 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:45.380043 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:45.380183 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:45.380183 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:45.380367 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:45.380416 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:45.380417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:45.381093 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:45.381094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:45.381609 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:45.381610 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:45.381961 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:45.381962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:45.382001 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:45.382440 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:45.382445 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:45.384352 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":83,"threads_started":1}
W20260808 16:57:45.384593 22144 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:45.379887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:50137 (local address 127.8.14.126:40981)
0808 16:57:45.379986 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:45.379988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:45.380026 (+ 38us) server_negotiation.cc:407] Connection header received
0808 16:57:45.380097 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:45.380097 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:45.380110 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:45.380166 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:45.380507 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:45.381014 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:45.381701 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:45.381892 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:45.381951 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:45.384503 (+ 2552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.67:50137: BlockingRecv error: recv got EOF from 127.8.14.67:50137 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 16:57:46 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:46 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:46.127163 22150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:46.122140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:60337)
0808 16:57:46.122385 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:46.122405 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:46.122466 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:46.122733 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:46.122734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:46.122978 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:46.123031 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:46.123032 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:46.123698 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:46.123699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:46.124227 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:46.124229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:46.124640 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:46.124641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:46.124711 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:46.125158 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:46.125163 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:46.127060 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":122,"threads_started":1}
W20260808 16:57:46.127226 22151 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:46.122419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:60337 (local address 127.8.14.126:40981)
0808 16:57:46.122593 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:46.122595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:46.122603 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:57:46.122633 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:46.122634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:46.122645 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:46.122710 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:46.123112 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:46.123628 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:46.124349 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:46.124558 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:46.124640 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:46.127152 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:60337: BlockingRecv error: recv got EOF from 127.8.14.65:60337 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":114,"threads_started":1}
Aug 08 16:57:46 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:46 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:46.257692 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:46.253099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:56661)
0808 16:57:46.253292 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:46.253303 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:46.253389 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:46.253558 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:46.253559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:46.253747 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:46.253797 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:46.253799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:46.254432 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:46.254433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:46.255011 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:46.255014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:46.255331 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:46.255332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:46.255373 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:46.255884 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:46.255890 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:46.257635 (+ 1745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":140,"threads_started":1}
W20260808 16:57:46.257836 22151 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:46.253137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:56661 (local address 127.8.14.126:40981)
0808 16:57:46.253192 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:46.253194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:46.253361 (+ 167us) server_negotiation.cc:407] Connection header received
0808 16:57:46.253462 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:46.253463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:46.253477 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:46.253535 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:46.253870 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:46.254356 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:46.255084 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:46.255266 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:46.255321 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:46.257739 (+ 2418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:56661: BlockingRecv error: recv got EOF from 127.8.14.66:56661 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 08 16:57:46 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:46 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:46.389495 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:46.385059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:34767)
0808 16:57:46.385235 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:46.385247 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:46.385310 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:46.385439 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:46.385440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:46.385609 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:46.385660 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:46.385662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:46.386276 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:46.386277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:46.386786 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:46.386787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:46.387116 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:46.387118 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:46.387174 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:46.387612 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:46.387616 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:46.389435 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":69,"threads_started":1}
W20260808 16:57:46.389684 22151 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:46.385137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:34767 (local address 127.8.14.126:40981)
0808 16:57:46.385239 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:46.385241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:46.385311 (+ 70us) server_negotiation.cc:407] Connection header received
0808 16:57:46.385353 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:46.385354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:46.385367 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:46.385416 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:46.385722 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:46.386199 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:46.386854 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:46.387050 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:46.387098 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:46.389551 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.67:34767: BlockingRecv error: recv got EOF from 127.8.14.67:34767 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 16:57:47 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:47 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:47.132686 22157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:47.127705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:48381)
0808 16:57:47.127853 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:47.127870 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:47.127934 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:47.128192 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:47.128193 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:47.128366 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:47.128416 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:47.128418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:47.129052 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:47.129052 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:47.129565 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:47.129567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:47.129890 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:47.129891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:47.129929 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:47.130413 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:47.130417 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:47.132584 (+ 2167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":61,"threads_started":1}
W20260808 16:57:47.132750 22158 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:47.127884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:48381 (local address 127.8.14.126:40981)
0808 16:57:47.128030 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:47.128032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:47.128041 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:57:47.128062 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:47.128063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:47.128073 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:47.128174 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:47.128495 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:47.128987 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:47.129638 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:47.129828 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:47.129880 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:47.132675 (+ 2795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:48381: BlockingRecv error: recv got EOF from 127.8.14.65:48381 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
Aug 08 16:57:47 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:47 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:47.262890 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:47.258216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:44075)
0808 16:57:47.258357 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:47.258371 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:47.258434 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:47.258575 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:47.258576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:47.258749 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:47.258811 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:47.258813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:47.259640 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:47.259640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:47.260158 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:47.260159 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:47.260492 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:47.260493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:47.260532 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:47.260972 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:47.260977 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:47.262794 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":95,"threads_started":1}
W20260808 16:57:47.262949 22158 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:47.258287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:44075 (local address 127.8.14.126:40981)
0808 16:57:47.258366 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:47.258368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:47.258434 (+ 66us) server_negotiation.cc:407] Connection header received
0808 16:57:47.258497 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:47.258498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:47.258512 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:47.258557 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:47.258877 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:47.259567 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:47.260232 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:47.260423 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:47.260476 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:47.262871 (+ 2395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:44075: BlockingRecv error: recv got EOF from 127.8.14.66:44075 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 16:57:47 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:47 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:47.395038 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:47.389995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:37491)
0808 16:57:47.390138 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:47.390153 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:47.390228 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:47.390376 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:47.390377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:47.390556 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:47.390629 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:47.390631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:47.391369 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:47.391370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:47.391916 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:47.391918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:47.392351 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:47.392353 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:47.392420 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:47.392968 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:47.392974 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:47.394941 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":59,"threads_started":1}
W20260808 16:57:47.395088 22158 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:47.390094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:37491 (local address 127.8.14.126:40981)
0808 16:57:47.390177 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:47.390179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:47.390210 (+ 31us) server_negotiation.cc:407] Connection header received
0808 16:57:47.390288 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:47.390289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:47.390301 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:47.390349 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:47.390713 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:47.391300 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:47.391985 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:47.392235 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:47.392291 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:47.395014 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.67:37491: BlockingRecv error: recv got EOF from 127.8.14.67:37491 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 16:57:48 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:48 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:48.138113 22161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:48.133260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:41037)
0808 16:57:48.133407 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:48.133418 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:48.133475 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:48.133622 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:48.133623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:48.133801 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:48.133850 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:48.133852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:48.134486 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:48.134487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:48.135099 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:48.135101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:48.135427 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:48.135428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:48.135466 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:48.135931 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:48.135935 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:48.138025 (+ 2090us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
W20260808 16:57:48.138201 22163 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:48.133278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:41037 (local address 127.8.14.126:40981)
0808 16:57:48.133411 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:48.133413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:48.133472 (+ 59us) server_negotiation.cc:407] Connection header received
0808 16:57:48.133521 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:48.133522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:48.133534 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:48.133605 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:48.133915 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:48.134415 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:48.135174 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:48.135366 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:48.135415 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:48.138127 (+ 2712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:41037: BlockingRecv error: recv got EOF from 127.8.14.65:41037 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
Aug 08 16:57:48 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:48 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:48.267938 22164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:48.263420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:34981)
0808 16:57:48.263554 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:48.263566 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:48.263626 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:48.263828 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:48.263828 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:48.264000 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:48.264045 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:48.264046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:48.264683 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:48.264684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:48.265199 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:48.265201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:48.265531 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:48.265531 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:48.265572 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:48.266051 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:48.266056 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:48.267880 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
W20260808 16:57:48.268062 22163 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:48.263629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:34981 (local address 127.8.14.126:40981)
0808 16:57:48.263696 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:48.263698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:48.263707 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:57:48.263739 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:48.263740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:48.263751 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:48.263810 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:48.264120 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:48.264615 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:48.265274 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:48.265465 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:48.265517 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:48.267964 (+ 2447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:34981: BlockingRecv error: recv got EOF from 127.8.14.66:34981 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 16:57:48 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:48 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:48.400309 22165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:48.395533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:55759)
0808 16:57:48.395697 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:48.395707 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:48.395762 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:48.395889 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:48.395890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:48.396060 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:48.396106 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:48.396108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:48.396795 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:48.396796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:48.397313 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:48.397315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:48.397679 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:48.397680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:48.397748 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:48.398235 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:48.398240 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:48.400203 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
W20260808 16:57:48.400344 22163 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:48.395583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:55759 (local address 127.8.14.126:40981)
0808 16:57:48.395675 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:48.395677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:48.395766 (+ 89us) server_negotiation.cc:407] Connection header received
0808 16:57:48.395803 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:48.395804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:48.395816 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:48.395875 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:48.396174 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:48.396719 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:48.397409 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:48.397603 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:48.397681 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:48.400281 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.67:55759: BlockingRecv error: recv got EOF from 127.8.14.67:55759 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 16:57:49 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:49 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:49.148010 22168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:49.138938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:58205)
0808 16:57:49.139086 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:49.139096 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:49.139157 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:49.139292 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:49.139293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:49.139473 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:49.139523 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:49.139524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:49.140496 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:49.140497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:49.141019 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:49.141020 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:49.141422 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:49.141422 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:49.141465 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:49.145960 (+ 4495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:49.145965 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:49.147937 (+ 1972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
W20260808 16:57:49.148150 22169 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:49.138969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:58205 (local address 127.8.14.126:40981)
0808 16:57:49.139102 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:49.139104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:49.139155 (+ 51us) server_negotiation.cc:407] Connection header received
0808 16:57:49.139201 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:49.139202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:49.139212 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:49.139278 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:49.139594 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:49.140424 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:49.141098 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:49.141355 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:49.141411 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:49.148068 (+ 6657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:58205: BlockingRecv error: recv got EOF from 127.8.14.65:58205 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
Aug 08 16:57:49 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:49 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:49.273350 22171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:49.268393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:45177)
0808 16:57:49.268575 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:49.268587 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:49.268680 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:49.268907 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:49.268908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:49.269151 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:49.269216 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:49.269218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:49.269931 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:49.269933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:49.270528 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:49.270529 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:49.270886 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:49.270887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:49.271012 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:49.271463 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:49.271469 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:49.273289 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":87,"threads_started":1}
W20260808 16:57:49.273447 22169 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:49.268494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:45177 (local address 127.8.14.126:40981)
0808 16:57:49.268570 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:49.268572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:49.268668 (+ 96us) server_negotiation.cc:407] Connection header received
0808 16:57:49.268789 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:49.268790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:49.268817 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:49.268888 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:49.269306 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:49.269823 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:49.270614 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:49.270813 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:49.270870 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:49.273382 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:45177: BlockingRecv error: recv got EOF from 127.8.14.66:45177 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 16:57:49 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:49 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:49.405695 22174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:49.400882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:37219)
0808 16:57:49.401033 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:49.401070 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:49.401136 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:49.401292 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:49.401293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:49.401489 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:49.401535 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:49.401537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:49.402213 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:49.402214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:49.402744 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:49.402746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:49.403221 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:49.403222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:49.403291 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:49.403792 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:49.403798 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:49.405599 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
W20260808 16:57:49.405747 22169 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:49.401024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:37219 (local address 127.8.14.126:40981)
0808 16:57:49.401109 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:49.401111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:49.401120 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:57:49.401212 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:49.401213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:49.401228 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:49.401275 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:49.401621 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:49.402125 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:49.402846 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:49.403142 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:49.403223 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:49.405680 (+ 2457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.67:37219: BlockingRecv error: recv got EOF from 127.8.14.67:37219 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 16:57:50 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:50 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:50.153206 22175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:50.148472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:60203)
0808 16:57:50.148622 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:50.148635 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:50.148702 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:50.148830 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:50.148831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:50.148999 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:50.149046 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:50.149047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:50.149673 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:50.149674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:50.150197 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:50.150199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:50.150687 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:50.150689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:50.150734 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:50.151231 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:50.151236 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:50.153133 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
W20260808 16:57:50.153348 22177 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:50.148518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:60203 (local address 127.8.14.126:40981)
0808 16:57:50.148667 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:50.148669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:50.148682 (+ 13us) server_negotiation.cc:407] Connection header received
0808 16:57:50.148749 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:50.148749 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:50.148761 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:50.148814 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:50.149105 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:50.149601 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:50.150289 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:50.150611 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:50.150677 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:50.153229 (+ 2552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:60203: BlockingRecv error: recv got EOF from 127.8.14.65:60203 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
Aug 08 16:57:50 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:50 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:50.278710 22178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:50.273862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:59999)
0808 16:57:50.274007 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:50.274025 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:50.274082 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:50.274236 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:50.274237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:50.274421 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:50.274468 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:50.274469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:50.275289 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:50.275289 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:50.275810 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:50.275812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:50.276236 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:50.276237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:50.276293 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:50.276742 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:50.276746 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:50.278641 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
W20260808 16:57:50.278786 22177 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:50.273960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:59999 (local address 127.8.14.126:40981)
0808 16:57:50.274011 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:50.274013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:50.274066 (+ 53us) server_negotiation.cc:407] Connection header received
0808 16:57:50.274149 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:50.274150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:50.274161 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:50.274217 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:50.274560 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:50.275222 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:50.275882 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:50.276124 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:50.276195 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:50.278740 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:59999: BlockingRecv error: recv got EOF from 127.8.14.66:59999 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 08 16:57:50 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:50 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:50.410812 22179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:50.406215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:43541)
0808 16:57:50.406391 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:50.406409 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:50.406470 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:50.406608 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:50.406609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:50.406792 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:50.406839 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:50.406841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:50.407494 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:50.407495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:50.408019 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:50.408020 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:50.408341 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:50.408342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:50.408381 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:50.408820 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:50.408824 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:50.410739 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":116,"threads_started":1}
W20260808 16:57:50.410887 22177 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:50.406304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:43541 (local address 127.8.14.126:40981)
0808 16:57:50.406376 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:50.406379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:50.406471 (+ 92us) server_negotiation.cc:407] Connection header received
0808 16:57:50.406516 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:50.406516 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:50.406529 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:50.406584 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:50.406924 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:50.407421 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:50.408091 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:50.408280 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:50.408327 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:50.410839 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.67:43541: BlockingRecv error: recv got EOF from 127.8.14.67:43541 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 16:57:51 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:51 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:51.158396 22181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:51.153716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:50173)
0808 16:57:51.153864 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:51.153875 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:51.153925 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:51.154124 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:51.154125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:51.154331 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:51.154378 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:51.154380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:51.155048 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:51.155049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:51.155638 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:51.155641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:51.155981 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:51.155982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:51.156025 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:51.156492 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:51.156496 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:51.158327 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
W20260808 16:57:51.158505 22182 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:51.153772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:50173 (local address 127.8.14.126:40981)
0808 16:57:51.153918 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:51.153920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:51.153931 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:57:51.153996 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:51.153997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:51.154010 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:51.154083 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:51.154449 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:51.154972 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:51.155717 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:51.155907 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:51.155957 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:51.158427 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:50173: BlockingRecv error: recv got EOF from 127.8.14.65:50173 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
Aug 08 16:57:51 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:51 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:51.284528 22184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:51.279435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:59005)
0808 16:57:51.279644 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:51.279660 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:51.279756 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:51.279919 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:51.279920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:51.280127 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:51.280175 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:51.280178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:51.280882 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:51.280884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:51.281411 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:51.281413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:51.281792 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:51.281794 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:51.281882 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:51.282429 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:51.282435 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:51.284452 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
W20260808 16:57:51.284605 22182 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:51.279576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:59005 (local address 127.8.14.126:40981)
0808 16:57:51.279676 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:51.279678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:51.279749 (+ 71us) server_negotiation.cc:407] Connection header received
0808 16:57:51.279821 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:51.279822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:51.279834 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:51.279912 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:51.280274 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:51.280789 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:51.281511 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:51.281711 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:51.281794 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:51.284557 (+ 2763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:59005: BlockingRecv error: recv got EOF from 127.8.14.66:59005 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 08 16:57:51 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:51 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:51.416517 22185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:51.411347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:57951)
0808 16:57:51.411490 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:51.411504 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:51.411580 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:51.411774 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:51.411775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:51.411979 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:51.412026 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:51.412027 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:51.412746 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:51.412747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:51.413368 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:51.413370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:51.413820 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:51.413821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:51.413885 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:51.414432 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:51.414437 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:51.416401 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
W20260808 16:57:51.416558 22182 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:51.411463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:57951 (local address 127.8.14.126:40981)
0808 16:57:51.411523 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:51.411526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:51.411559 (+ 33us) server_negotiation.cc:407] Connection header received
0808 16:57:51.411666 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:51.411667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:51.411689 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:51.411751 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:51.412124 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:51.412655 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:51.413460 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:51.413734 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:51.413801 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:51.416491 (+ 2690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.67:57951: BlockingRecv error: recv got EOF from 127.8.14.67:57951 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 08 16:57:52 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:52 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:52.163760 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:52.159019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:39973)
0808 16:57:52.159173 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:52.159186 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:52.159264 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:52.159417 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:52.159417 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:52.159592 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:52.159655 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:52.159656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:52.160379 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:52.160380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:52.160914 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:52.160915 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:52.161317 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:52.161318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:52.161383 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:52.161795 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:52.161800 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:52.163688 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":60,"threads_started":1}
W20260808 16:57:52.163897 22192 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:52.159062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:39973 (local address 127.8.14.126:40981)
0808 16:57:52.159268 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:52.159270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:52.159280 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:57:52.159328 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:52.159328 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:52.159340 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:52.159398 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:52.159745 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:52.160281 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:52.161006 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:52.161228 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:52.161304 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:52.163799 (+ 2495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:39973: BlockingRecv error: recv got EOF from 127.8.14.65:39973 (error 108)
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":126,"threads_started":1}
Aug 08 16:57:52 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:52 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:52.290678 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:52.285215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:47671)
0808 16:57:52.285374 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:52.285389 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:52.285493 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:52.285712 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:52.285713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:52.285921 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:52.285969 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:52.285972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:52.286747 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:52.286749 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:52.287458 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:52.287461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:52.287889 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:52.287891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:52.287959 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:52.288514 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:52.288520 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:52.290601 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
W20260808 16:57:52.290745 22192 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:52.285333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:47671 (local address 127.8.14.126:40981)
0808 16:57:52.285394 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:52.285396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:52.285477 (+ 81us) server_negotiation.cc:407] Connection header received
0808 16:57:52.285584 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:52.285585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:52.285601 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:52.285679 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:52.286062 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:52.286645 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:52.287570 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:52.287791 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:52.287886 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:52.290697 (+ 2811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:47671: BlockingRecv error: recv got EOF from 127.8.14.66:47671 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 16:57:52 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:52 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:52.421872 22196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:52.417055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:46127)
0808 16:57:52.417254 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:52.417265 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:52.417325 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:52.417454 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:52.417455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:52.417637 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:52.417684 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:52.417686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:52.418446 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:52.418447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:52.419059 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:52.419062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:52.419463 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:52.419464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:52.419521 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:52.419986 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:52.419991 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:52.421806 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":138,"threads_started":1}
W20260808 16:57:52.422003 22192 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:52.417110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:46127 (local address 127.8.14.126:40981)
0808 16:57:52.417176 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:52.417178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:52.417324 (+ 146us) server_negotiation.cc:407] Connection header received
0808 16:57:52.417367 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:52.417368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:52.417382 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:52.417432 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:52.417774 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:52.418365 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:52.419162 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:52.419347 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:52.419464 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:52.421924 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.67:46127: BlockingRecv error: recv got EOF from 127.8.14.67:46127 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 16:57:53 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:53 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:53.169044 22197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:53.164257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:46813)
0808 16:57:53.164415 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:53.164426 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:53.164485 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:53.164639 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:53.164640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:53.164842 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:53.164890 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:53.164892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:53.165522 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:53.165523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:53.166046 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:53.166047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:53.166469 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:53.166470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:53.166508 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:53.167060 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:53.167065 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:53.168943 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
W20260808 16:57:53.169101 22200 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:53.164341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:46813 (local address 127.8.14.126:40981)
0808 16:57:53.164472 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:53.164474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:53.164485 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:57:53.164549 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:53.164549 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:53.164561 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:53.164626 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:53.164954 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:53.165456 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:53.166145 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:53.166395 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:53.166461 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:53.169027 (+ 2566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:46813: BlockingRecv error: recv got EOF from 127.8.14.65:46813 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
Aug 08 16:57:53 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:53 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:53.296307 22201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:53.291264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:60981)
0808 16:57:53.291433 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:53.291444 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:53.291532 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:53.291697 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:53.291698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:53.291874 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:53.291922 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:53.291924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:53.292622 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:53.292623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:53.293256 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:53.293260 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:53.293738 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:53.293739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:53.293798 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:53.294327 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:53.294331 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:53.296235 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
W20260808 16:57:53.296412 22200 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:53.291313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:60981 (local address 127.8.14.126:40981)
0808 16:57:53.291376 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:53.291378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:53.291500 (+ 122us) server_negotiation.cc:407] Connection header received
0808 16:57:53.291615 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:53.291616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:53.291627 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:53.291689 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:53.291981 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:53.292533 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:53.293370 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:53.293673 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:53.293739 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:53.296340 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:60981: BlockingRecv error: recv got EOF from 127.8.14.66:60981 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 08 16:57:53 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:53 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.67@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:53.427763 22206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:53.422355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.67:37815)
0808 16:57:53.422538 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:53.422550 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:53.422607 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:53.422740 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:53.422741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:53.423067 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:53.423117 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:53.423119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:53.423806 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:53.423806 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:53.424352 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:53.424353 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:53.424730 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:53.424731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:53.424809 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:53.425399 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:53.425405 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:53.427637 (+ 2232us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":118,"threads_started":1}
W20260808 16:57:53.427809 22200 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:53.422407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:37815 (local address 127.8.14.126:40981)
0808 16:57:53.422478 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:53.422481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:53.422609 (+ 128us) server_negotiation.cc:407] Connection header received
0808 16:57:53.422661 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:53.422662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:53.422673 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:53.422732 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:53.423183 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:53.423737 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:53.424424 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:53.424636 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:53.424727 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:53.427736 (+ 3009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.67:37815: BlockingRecv error: recv got EOF from 127.8.14.67:37815 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 16:57:54 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:54 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.65@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:54.178339 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:54.169642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.65:60027)
0808 16:57:54.169821 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:54.169832 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:54.169890 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:54.170033 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:54.170034 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:54.170197 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:54.170248 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:54.170250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.170940 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.170941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:54.171470 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:54.171472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.171821 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.171822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:54.171862 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:54.176369 (+ 4507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:54.176375 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:54.178224 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":101,"threads_started":1}
W20260808 16:57:54.178386 22209 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:54.169751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:60027 (local address 127.8.14.126:40981)
0808 16:57:54.169875 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:54.169877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:54.169898 (+ 21us) server_negotiation.cc:407] Connection header received
0808 16:57:54.169930 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:54.169931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:54.169955 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:54.170014 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:54.170315 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.170834 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.171547 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.171748 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.171803 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:54.178310 (+ 6507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.65:60027: BlockingRecv error: recv got EOF from 127.8.14.65:60027 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
Aug 08 16:57:54 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 16:57:54 dist-test-slave-1nl6 krb5kdc[20465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.14.66@KRBTEST.COM for oryx/127.8.14.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 16:57:54.301213 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:54.296947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:40981 (local address 127.8.14.66:35571)
0808 16:57:54.297113 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:54.297124 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:54.297183 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:54.297328 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:54.297329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:54.297496 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:54.297545 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:54.297547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.298161 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.298162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:54.298677 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:54.298678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.299004 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.299005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:54.299047 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:54.299476 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:54.299481 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:54.301155 (+ 1674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.14.126:40981: Server oryx/127.8.14.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":100,"threads_started":1}
W20260808 16:57:54.301339 22209 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:54.297044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:35571 (local address 127.8.14.126:40981)
0808 16:57:54.297107 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:54.297109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:54.297186 (+ 77us) server_negotiation.cc:407] Connection header received
0808 16:57:54.297242 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:54.297242 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:54.297254 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:54.297311 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:54.297607 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.298093 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.298744 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.298943 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.298995 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:54.301261 (+ 2266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.14.66:35571: BlockingRecv error: recv got EOF from 127.8.14.66:35571 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 16:57:54.347581 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 21424
I20260808 16:57:54.359014 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 21684
I20260808 16:57:54.367024 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 21889
I20260808 16:57:54.374267 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 21309
2026-08-08T16:57:54Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17728 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T16:57:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:57:54Z Disabled control of system clock
I20260808 16:57:54.406843 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:37839
--webserver_interface=127.8.14.126
--webserver_port=0
--builtin_ntp_servers=127.8.14.84:39509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:37839
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 16:57:54.488680 22216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:54.488932 22216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:54.488984 22216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:54.490185 22216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:57:54.490253 22216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:54.490275 22216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:54.490296 22216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:57:54.490327 22216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:57:54.491705 22216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:37839
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:37839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:54.491979 22216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:54.492295 22216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:54.497529 22229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:54.497605 22225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:54.497553 22227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:54.497627 22216 server_base.cc:1034] running on GCE node
I20260808 16:57:54.497951 22216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:54.498252 22216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:54.499392 22216 hybrid_clock.cc:648] HybridClock initialized: now 1786208274499380 us; error 28 us; skew 500 ppm
I20260808 16:57:54.500563 22216 webserver.cc:460] Webserver started at http://127.8.14.126:38919/ using document root <none> and password file <none>
I20260808 16:57:54.500768 22216 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:54.500809 22216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:54.500901 22216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:54.501951 22216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "00b4c125393e4a3e9c342d2b8b496406"
format_stamp: "Formatted at 2026-08-08 16:57:54 on dist-test-slave-1nl6"
server_key: "d287952e77d655fec3b78472b93fcdfd"
server_key_iv: "067784be45391196a5bef53b9a7d1471"
server_key_version: "encryptionkey@0"
I20260808 16:57:54.502255 22216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "00b4c125393e4a3e9c342d2b8b496406"
format_stamp: "Formatted at 2026-08-08 16:57:54 on dist-test-slave-1nl6"
server_key: "d287952e77d655fec3b78472b93fcdfd"
server_key_iv: "067784be45391196a5bef53b9a7d1471"
server_key_version: "encryptionkey@0"
I20260808 16:57:54.503623 22216 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:57:54.504344 22237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:54.504522 22216 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:57:54.504631 22216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "00b4c125393e4a3e9c342d2b8b496406"
format_stamp: "Formatted at 2026-08-08 16:57:54 on dist-test-slave-1nl6"
server_key: "d287952e77d655fec3b78472b93fcdfd"
server_key_iv: "067784be45391196a5bef53b9a7d1471"
server_key_version: "encryptionkey@0"
I20260808 16:57:54.504720 22216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:54.525193 22216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:54.525475 22216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:54.525621 22216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:54.529824 22216 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:37839
I20260808 16:57:54.529857 22325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:37839 every 8 connection(s)
I20260808 16:57:54.530229 22216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:57:54.530956 22326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:54.531613 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 22216
I20260808 16:57:54.531720 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:57:54.531934 8249 external_mini_cluster.cc:1428] Setting key f8adbf045dfc7fd4e99dae589315e7d7
I20260808 16:57:54.534438 22326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406: Bootstrap starting.
I20260808 16:57:54.535163 22326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:54.535490 22326 log.cc:824] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406: Log is configured to *not* fsync() on all Append() calls
I20260808 16:57:54.536283 22326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406: No bootstrap required, opened a new log
I20260808 16:57:54.538137 22326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b4c125393e4a3e9c342d2b8b496406" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37839 } }
I20260808 16:57:54.538272 22326 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:54.538306 22330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:54.532489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41458 (local address 127.8.14.126:37839)
0808 16:57:54.532714 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:54.532723 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:54.532757 (+ 34us) server_negotiation.cc:407] Connection header received
0808 16:57:54.533324 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:54.533333 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:54.533410 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:54.533566 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:54.533842 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.535190 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.536223 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.536656 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.536746 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:54.537425 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:54.537429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:54.537429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:57:54.537587 (+ 158us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:54.537619 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:57:54.537628 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:54.537682 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:54.537789 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 16:57:54.537876 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260808 16:57:54.538295 22326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00b4c125393e4a3e9c342d2b8b496406, State: Initialized, Role: FOLLOWER
I20260808 16:57:54.538506 22326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b4c125393e4a3e9c342d2b8b496406" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37839 } }
I20260808 16:57:54.538597 22326 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:57:54.538621 22326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:57:54.538691 22326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:54.539357 22326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b4c125393e4a3e9c342d2b8b496406" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37839 } }
I20260808 16:57:54.539458 22326 leader_election.cc:302] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00b4c125393e4a3e9c342d2b8b496406; no voters:
I20260808 16:57:54.539662 22326 leader_election.cc:288] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:57:54.539747 22332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:57:54.539909 22332 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [term 1 LEADER]: Becoming Leader. State: Replica: 00b4c125393e4a3e9c342d2b8b496406, State: Running, Role: LEADER
I20260808 16:57:54.539963 22326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:57:54.540053 22332 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b4c125393e4a3e9c342d2b8b496406" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37839 } }
I20260808 16:57:54.540478 22334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00b4c125393e4a3e9c342d2b8b496406. Latest consensus state: current_term: 1 leader_uuid: "00b4c125393e4a3e9c342d2b8b496406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b4c125393e4a3e9c342d2b8b496406" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37839 } } }
I20260808 16:57:54.540562 22334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [sys.catalog]: This master's current role is: LEADER
I20260808 16:57:54.540776 22333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00b4c125393e4a3e9c342d2b8b496406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b4c125393e4a3e9c342d2b8b496406" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37839 } } }
I20260808 16:57:54.541052 22333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406 [sys.catalog]: This master's current role is: LEADER
I20260808 16:57:54.541193 22346 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:57:54.541663 22346 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:57:54.543224 22346 catalog_manager.cc:1347] Generated new cluster ID: ecf6b079b48a4275995020d1cb87da5e
I20260808 16:57:54.543278 22346 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:57:54.560686 22346 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:57:54.561194 22346 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:57:54.566011 22346 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00b4c125393e4a3e9c342d2b8b496406: Generated new TSK 0
I20260808 16:57:54.566185 22346 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:57:54.570703 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:37839
--builtin_ntp_servers=127.8.14.84:39509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 16:57:54.655267 22362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:54.655493 22362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:54.655526 22362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:54.656637 22362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:54.656701 22362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:54.656744 22362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:57:54.658039 22362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:37839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:54.658308 22362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:54.658595 22362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:54.664542 22376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:54.664564 22378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:54.664527 22375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:54.664971 22362 server_base.cc:1034] running on GCE node
I20260808 16:57:54.665184 22362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:54.665436 22362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:54.666591 22362 hybrid_clock.cc:648] HybridClock initialized: now 1786208274666573 us; error 34 us; skew 500 ppm
I20260808 16:57:54.667891 22362 webserver.cc:460] Webserver started at http://127.8.14.65:34231/ using document root <none> and password file <none>
I20260808 16:57:54.668078 22362 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:54.668118 22362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:54.668218 22362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:54.669210 22362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "f6c4732bb6e74e3fbeb1be8f0ca7d51d"
format_stamp: "Formatted at 2026-08-08 16:57:54 on dist-test-slave-1nl6"
server_key: "30de53d385b0d558def3045c89c6bee7"
server_key_iv: "7a82ab7122b8c5180aafbb3a84ef240c"
server_key_version: "encryptionkey@0"
I20260808 16:57:54.669508 22362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "f6c4732bb6e74e3fbeb1be8f0ca7d51d"
format_stamp: "Formatted at 2026-08-08 16:57:54 on dist-test-slave-1nl6"
server_key: "30de53d385b0d558def3045c89c6bee7"
server_key_iv: "7a82ab7122b8c5180aafbb3a84ef240c"
server_key_version: "encryptionkey@0"
I20260808 16:57:54.670744 22362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:57:54.671553 22385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:54.671725 22362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:54.671818 22362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "f6c4732bb6e74e3fbeb1be8f0ca7d51d"
format_stamp: "Formatted at 2026-08-08 16:57:54 on dist-test-slave-1nl6"
server_key: "30de53d385b0d558def3045c89c6bee7"
server_key_iv: "7a82ab7122b8c5180aafbb3a84ef240c"
server_key_version: "encryptionkey@0"
I20260808 16:57:54.671893 22362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:54.686656 22362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:54.687012 22362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:54.687170 22362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:54.687402 22362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:54.687734 22362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:54.687769 22362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:54.687790 22362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:54.687845 22362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:54.693616 22362 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:34069
I20260808 16:57:54.693668 22570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:34069 every 8 connection(s)
I20260808 16:57:54.694002 22362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:57:54.696561 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 22362
I20260808 16:57:54.696640 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:57:54.696844 8249 external_mini_cluster.cc:1428] Setting key 1af479f9af9aff72f4d92e76a3ec94cd
I20260808 16:57:54.697815 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:37839
--builtin_ntp_servers=127.8.14.84:39509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 16:57:54.699677 22330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:54.694736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:49453 (local address 127.8.14.126:37839)
0808 16:57:54.694802 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:54.694805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:54.695158 (+ 353us) server_negotiation.cc:407] Connection header received
0808 16:57:54.695617 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:54.695618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:54.695629 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:54.695682 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:54.696307 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.696926 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.698157 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.698371 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.698446 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:54.699391 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:54.699393 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:54.699394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:57:54.699400 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:54.699419 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:57:54.699424 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:54.699490 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:54.699593 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 16:57:54.699620 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 16:57:54.699954 22574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:54.694743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:37839 (local address 127.8.14.65:49453)
0808 16:57:54.694998 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:54.695044 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:54.695533 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:54.695743 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:54.695746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:54.695754 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:54.696235 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:54.696239 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.697016 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.697017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:54.698068 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:54.698071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.698457 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.698458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:54.698596 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:54.699184 (+ 588us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:57:54.699185 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:54.699212 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:57:54.699256 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:54.699502 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:54.699509 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:54.699512 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:57:54.699582 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 16:57:54.699702 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":92,"threads_started":1}
I20260808 16:57:54.700506 22571 heartbeater.cc:347] Connected to a master server at 127.8.14.126:37839
I20260808 16:57:54.700613 22571 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:54.700799 22571 heartbeater.cc:511] Master 127.8.14.126:37839 requested a full tablet report, sending...
I20260808 16:57:54.701313 22266 ts_manager.cc:194] Registered new tserver with Master: f6c4732bb6e74e3fbeb1be8f0ca7d51d (127.8.14.65:34069)
I20260808 16:57:54.701952 22266 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.14.65:49453
W20260808 16:57:54.781486 22575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:54.781749 22575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:54.781787 22575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:54.782956 22575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:54.783017 22575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:54.783057 22575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:57:54.784399 22575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:37839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:54.784672 22575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:54.784974 22575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:54.790517 22585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:54.790506 22586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:54.790617 22575 server_base.cc:1034] running on GCE node
W20260808 16:57:54.790514 22588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:54.791059 22575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:54.791321 22575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:54.792488 22575 hybrid_clock.cc:648] HybridClock initialized: now 1786208274792474 us; error 27 us; skew 500 ppm
I20260808 16:57:54.793751 22575 webserver.cc:460] Webserver started at http://127.8.14.66:44547/ using document root <none> and password file <none>
I20260808 16:57:54.794039 22575 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:54.794107 22575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:54.794245 22575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:54.795396 22575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "65c1d3ecea8743df95e379cf4df4c7f0"
format_stamp: "Formatted at 2026-08-08 16:57:54 on dist-test-slave-1nl6"
server_key: "cd1d6874f0d7d34a215d31ea9c822756"
server_key_iv: "a0f9c45208e6bc579aea80f1bc98a4f8"
server_key_version: "encryptionkey@0"
I20260808 16:57:54.795778 22575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "65c1d3ecea8743df95e379cf4df4c7f0"
format_stamp: "Formatted at 2026-08-08 16:57:54 on dist-test-slave-1nl6"
server_key: "cd1d6874f0d7d34a215d31ea9c822756"
server_key_iv: "a0f9c45208e6bc579aea80f1bc98a4f8"
server_key_version: "encryptionkey@0"
I20260808 16:57:54.797111 22575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:57:54.797865 22601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:54.798038 22575 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:57:54.798142 22575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "65c1d3ecea8743df95e379cf4df4c7f0"
format_stamp: "Formatted at 2026-08-08 16:57:54 on dist-test-slave-1nl6"
server_key: "cd1d6874f0d7d34a215d31ea9c822756"
server_key_iv: "a0f9c45208e6bc579aea80f1bc98a4f8"
server_key_version: "encryptionkey@0"
I20260808 16:57:54.798234 22575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:54.822160 22575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:54.822440 22575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:54.822592 22575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:54.822822 22575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:54.823156 22575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:54.823220 22575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:54.823257 22575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:54.823285 22575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:54.828837 22575 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:34475
I20260808 16:57:54.828874 22790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:34475 every 8 connection(s)
I20260808 16:57:54.829185 22575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:57:54.834074 22330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:54.829923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:52791 (local address 127.8.14.126:37839)
0808 16:57:54.830002 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:54.830005 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:54.830202 (+ 197us) server_negotiation.cc:407] Connection header received
0808 16:57:54.830676 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:54.830677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:54.830687 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:54.830729 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:54.831428 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.831905 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.832878 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.833058 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.833113 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:54.833838 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:54.833839 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:54.833839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:57:54.833843 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:54.833855 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:57:54.833859 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:54.833902 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:54.833994 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 16:57:54.834018 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 16:57:54.834290 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:54.829876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:37839 (local address 127.8.14.66:52791)
0808 16:57:54.830081 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:54.830120 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:54.830574 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:54.830780 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:54.830783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:54.830790 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:54.831354 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:54.831359 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.831997 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.831998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:54.832798 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:54.832801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.833162 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.833163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:54.833276 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:54.833716 (+ 440us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:57:54.833716 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:54.833742 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:57:54.833763 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:54.833922 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:54.833926 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:54.833928 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:57:54.833985 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 16:57:54.834058 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260808 16:57:54.834613 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 22575
I20260808 16:57:54.834730 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:57:54.834939 22791 heartbeater.cc:347] Connected to a master server at 127.8.14.126:37839
I20260808 16:57:54.834964 8249 external_mini_cluster.cc:1428] Setting key e737425edafdf9600b771bc0b6a80d7c
I20260808 16:57:54.835029 22791 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:54.835220 22791 heartbeater.cc:511] Master 127.8.14.126:37839 requested a full tablet report, sending...
I20260808 16:57:54.835584 22266 ts_manager.cc:194] Registered new tserver with Master: 65c1d3ecea8743df95e379cf4df4c7f0 (127.8.14.66:34475)
I20260808 16:57:54.835871 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:37839
--builtin_ntp_servers=127.8.14.84:39509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 16:57:54.835978 22266 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.14.66:52791
W20260808 16:57:54.918004 22795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:54.918253 22795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:54.918295 22795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:54.919481 22795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:54.919538 22795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:54.919575 22795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:57:54.920882 22795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:37839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:54.921132 22795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:54.921435 22795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:54.926394 22804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:54.926383 22803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:54.926381 22807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:54.926610 22795 server_base.cc:1034] running on GCE node
I20260808 16:57:54.926864 22795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:54.927142 22795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:54.928299 22795 hybrid_clock.cc:648] HybridClock initialized: now 1786208274928289 us; error 30 us; skew 500 ppm
I20260808 16:57:54.929483 22795 webserver.cc:460] Webserver started at http://127.8.14.67:46371/ using document root <none> and password file <none>
I20260808 16:57:54.929728 22795 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:54.929796 22795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:54.929935 22795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:54.931012 22795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "ebf0d4b42cfe4deab34ddc15a6ea619b"
format_stamp: "Formatted at 2026-08-08 16:57:54 on dist-test-slave-1nl6"
server_key: "e98cd15558a10aaf0d66e5a8162e9ed5"
server_key_iv: "8d8c313f6296bd4b827bccfa790bdcd5"
server_key_version: "encryptionkey@0"
I20260808 16:57:54.931375 22795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "ebf0d4b42cfe4deab34ddc15a6ea619b"
format_stamp: "Formatted at 2026-08-08 16:57:54 on dist-test-slave-1nl6"
server_key: "e98cd15558a10aaf0d66e5a8162e9ed5"
server_key_iv: "8d8c313f6296bd4b827bccfa790bdcd5"
server_key_version: "encryptionkey@0"
I20260808 16:57:54.932673 22795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:57:54.933377 22817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:54.933552 22795 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:57:54.933650 22795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "ebf0d4b42cfe4deab34ddc15a6ea619b"
format_stamp: "Formatted at 2026-08-08 16:57:54 on dist-test-slave-1nl6"
server_key: "e98cd15558a10aaf0d66e5a8162e9ed5"
server_key_iv: "8d8c313f6296bd4b827bccfa790bdcd5"
server_key_version: "encryptionkey@0"
I20260808 16:57:54.933740 22795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:54.943783 22795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:54.944044 22795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:54.944183 22795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:54.944438 22795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:54.944747 22795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:54.944782 22795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:54.944801 22795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:54.944813 22795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:54.950335 22795 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:37557
I20260808 16:57:54.950373 23004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:37557 every 8 connection(s)
I20260808 16:57:54.950681 22795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 16:57:54.951345 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 22795
I20260808 16:57:54.951440 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:57:54.951658 8249 external_mini_cluster.cc:1428] Setting key c3a6fb7f728b2085274ccf823c04b4ff
I20260808 16:57:54.955963 22330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:54.951440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:41749 (local address 127.8.14.126:37839)
0808 16:57:54.951487 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:54.951489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:54.951768 (+ 279us) server_negotiation.cc:407] Connection header received
0808 16:57:54.952389 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:54.952390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:54.952401 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:54.952461 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:54.953087 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.953578 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.954550 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.954746 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.954825 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:54.955690 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:54.955691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:54.955691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:57:54.955696 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:54.955717 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:57:54.955721 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:54.955778 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:54.955863 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 16:57:54.955887 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:57:54.956178 23008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:54.951427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:37839 (local address 127.8.14.67:41749)
0808 16:57:54.951634 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:54.951664 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:54.952295 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:54.952526 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:54.952530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:54.952539 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:54.952993 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:54.952997 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.953679 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.953680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:54.954444 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:54.954446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:54.954832 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:54.954833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:54.955001 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:54.955524 (+ 523us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:57:54.955524 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:54.955551 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:57:54.955571 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:54.955793 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:54.955799 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:54.955802 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:57:54.955865 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 16:57:54.955960 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
I20260808 16:57:54.956585 23006 heartbeater.cc:347] Connected to a master server at 127.8.14.126:37839
I20260808 16:57:54.956677 23006 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:54.956804 23006 heartbeater.cc:511] Master 127.8.14.126:37839 requested a full tablet report, sending...
I20260808 16:57:54.957130 22266 ts_manager.cc:194] Registered new tserver with Master: ebf0d4b42cfe4deab34ddc15a6ea619b (127.8.14.67:37557)
I20260808 16:57:54.957504 22266 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.14.67:41749
I20260808 16:57:54.963375 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 16:57:54.965209 22330 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:54.964716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41466 (local address 127.8.14.126:37839)
0808 16:57:54.964797 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:54.964799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:54.964899 (+ 100us) server_negotiation.cc:407] Connection header received
0808 16:57:54.964965 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:54.964966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:54.964976 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:54.965031 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:54.965155 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41466: BlockingRecv error: recv got EOF from 127.0.0.1:41466 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 16:57:54.965323 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 22362
I20260808 16:57:54.979969 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 22575
I20260808 16:57:54.993438 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 22795
I20260808 16:57:55.002053 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 22216
2026-08-08T16:57:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (629 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T16:57:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:57:55Z Disabled control of system clock
I20260808 16:57:55.036181 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:39249
--webserver_interface=127.8.14.126
--webserver_port=0
--builtin_ntp_servers=127.8.14.84:36947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:39249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 16:57:55.116838 23030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:55.117033 23030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:55.117053 23030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:55.118091 23030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:57:55.118132 23030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:55.118146 23030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:55.118160 23030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:57:55.118171 23030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:57:55.119499 23030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:36947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:39249
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:39249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:55.119695 23030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:55.119939 23030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:55.124562 23044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:55.124579 23046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:55.124629 23041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:55.124779 23030 server_base.cc:1034] running on GCE node
I20260808 16:57:55.125005 23030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:55.125260 23030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:55.126410 23030 hybrid_clock.cc:648] HybridClock initialized: now 1786208275126393 us; error 29 us; skew 500 ppm
I20260808 16:57:55.127739 23030 webserver.cc:460] Webserver started at http://127.8.14.126:40275/ using document root <none> and password file <none>
I20260808 16:57:55.127978 23030 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:55.128039 23030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:55.128219 23030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:55.129350 23030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "8f82e3950b8544d4b064ac282177af79"
format_stamp: "Formatted at 2026-08-08 16:57:55 on dist-test-slave-1nl6"
server_key: "95daec83193d1123b743269c1e080bbd"
server_key_iv: "0e64ca5f11d42766a5c7db991fd4ac45"
server_key_version: "encryptionkey@0"
I20260808 16:57:55.129662 23030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "8f82e3950b8544d4b064ac282177af79"
format_stamp: "Formatted at 2026-08-08 16:57:55 on dist-test-slave-1nl6"
server_key: "95daec83193d1123b743269c1e080bbd"
server_key_iv: "0e64ca5f11d42766a5c7db991fd4ac45"
server_key_version: "encryptionkey@0"
I20260808 16:57:55.130816 23030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:57:55.131558 23056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:55.131744 23030 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:57:55.131841 23030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "8f82e3950b8544d4b064ac282177af79"
format_stamp: "Formatted at 2026-08-08 16:57:55 on dist-test-slave-1nl6"
server_key: "95daec83193d1123b743269c1e080bbd"
server_key_iv: "0e64ca5f11d42766a5c7db991fd4ac45"
server_key_version: "encryptionkey@0"
I20260808 16:57:55.131916 23030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:55.135226 23030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:55.135478 23030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:55.135614 23030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:55.139611 23030 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:39249
I20260808 16:57:55.139639 23151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:39249 every 8 connection(s)
I20260808 16:57:55.139954 23030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:57:55.140453 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 23030
I20260808 16:57:55.140553 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:57:55.140627 23152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:55.140781 8249 external_mini_cluster.cc:1428] Setting key bff0c6a933173b099d690cb634222197
I20260808 16:57:55.143669 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:55.141327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33736 (local address 127.8.14.126:39249)
0808 16:57:55.141510 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:55.141520 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:55.141549 (+ 29us) server_negotiation.cc:407] Connection header received
0808 16:57:55.142274 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:55.142281 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:55.142322 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:55.142438 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:55.143198 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:55.143202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:55.143202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:57:55.143206 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:55.143245 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:57:55.143254 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:55.143311 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:55.143404 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 16:57:55.143473 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260808 16:57:55.144455 23152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79: Bootstrap starting.
I20260808 16:57:55.145028 23152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:55.145329 23152 log.cc:824] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79: Log is configured to *not* fsync() on all Append() calls
I20260808 16:57:55.146020 23152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79: No bootstrap required, opened a new log
I20260808 16:57:55.147416 23152 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f82e3950b8544d4b064ac282177af79" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 39249 } }
I20260808 16:57:55.147572 23152 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:55.147639 23152 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f82e3950b8544d4b064ac282177af79, State: Initialized, Role: FOLLOWER
I20260808 16:57:55.147785 23152 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f82e3950b8544d4b064ac282177af79" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 39249 } }
I20260808 16:57:55.147897 23152 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:57:55.147945 23152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:57:55.147994 23152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:55.148491 23152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f82e3950b8544d4b064ac282177af79" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 39249 } }
I20260808 16:57:55.148612 23152 leader_election.cc:302] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f82e3950b8544d4b064ac282177af79; no voters:
I20260808 16:57:55.148792 23152 leader_election.cc:288] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:57:55.149055 23152 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:57:55.149078 23163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:57:55.149219 23163 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [term 1 LEADER]: Becoming Leader. State: Replica: 8f82e3950b8544d4b064ac282177af79, State: Running, Role: LEADER
I20260808 16:57:55.149345 23163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f82e3950b8544d4b064ac282177af79" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 39249 } }
I20260808 16:57:55.149806 23166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f82e3950b8544d4b064ac282177af79. Latest consensus state: current_term: 1 leader_uuid: "8f82e3950b8544d4b064ac282177af79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f82e3950b8544d4b064ac282177af79" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 39249 } } }
I20260808 16:57:55.149811 23164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f82e3950b8544d4b064ac282177af79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f82e3950b8544d4b064ac282177af79" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 39249 } } }
I20260808 16:57:55.149919 23166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [sys.catalog]: This master's current role is: LEADER
I20260808 16:57:55.149930 23164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79 [sys.catalog]: This master's current role is: LEADER
I20260808 16:57:55.150103 23177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:57:55.150581 23177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:57:55.152061 23177 catalog_manager.cc:1347] Generated new cluster ID: de21f412654746009ec3b6ef3f5b6dac
I20260808 16:57:55.152148 23177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:57:55.172466 23177 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:57:55.172642 23177 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:57:55.176540 23177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f82e3950b8544d4b064ac282177af79: Generated new TSK 0
I20260808 16:57:55.176723 23177 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:57:55.179669 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:39249
--builtin_ntp_servers=127.8.14.84:36947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 16:57:55.266688 23192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:55.266886 23192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:55.266992 23192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:55.268122 23192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:55.268162 23192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:55.268186 23192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:57:55.269508 23192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:36947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:39249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:55.269732 23192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:55.270032 23192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:55.276026 23202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:55.276059 23200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:55.276249 23192 server_base.cc:1034] running on GCE node
W20260808 16:57:55.276304 23199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:55.276539 23192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:55.276815 23192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:55.277974 23192 hybrid_clock.cc:648] HybridClock initialized: now 1786208275277964 us; error 28 us; skew 500 ppm
I20260808 16:57:55.279356 23192 webserver.cc:460] Webserver started at http://127.8.14.65:38453/ using document root <none> and password file <none>
I20260808 16:57:55.279597 23192 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:55.279642 23192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:55.279786 23192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:55.280840 23192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "043fe230c2f14fcf9723f323a4f04c12"
format_stamp: "Formatted at 2026-08-08 16:57:55 on dist-test-slave-1nl6"
server_key: "169ce0e19713503ddd9d36ff26618b65"
server_key_iv: "b522135d0b9410ed68166cf360f74488"
server_key_version: "encryptionkey@0"
I20260808 16:57:55.281206 23192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "043fe230c2f14fcf9723f323a4f04c12"
format_stamp: "Formatted at 2026-08-08 16:57:55 on dist-test-slave-1nl6"
server_key: "169ce0e19713503ddd9d36ff26618b65"
server_key_iv: "b522135d0b9410ed68166cf360f74488"
server_key_version: "encryptionkey@0"
I20260808 16:57:55.282574 23192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:57:55.283437 23210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:55.283612 23192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 16:57:55.283721 23192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "043fe230c2f14fcf9723f323a4f04c12"
format_stamp: "Formatted at 2026-08-08 16:57:55 on dist-test-slave-1nl6"
server_key: "169ce0e19713503ddd9d36ff26618b65"
server_key_iv: "b522135d0b9410ed68166cf360f74488"
server_key_version: "encryptionkey@0"
I20260808 16:57:55.283802 23192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:55.287036 23192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:55.287295 23192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:55.287438 23192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:55.287655 23192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:55.287973 23192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:55.288008 23192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:55.288061 23192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:55.288089 23192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:55.293910 23192 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:37943
I20260808 16:57:55.293970 23394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:37943 every 8 connection(s)
I20260808 16:57:55.294253 23192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:57:55.295848 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 23192
I20260808 16:57:55.295949 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:57:55.296151 8249 external_mini_cluster.cc:1428] Setting key 3cb6cacbbd397a17f7b71cd50c4ba14f
I20260808 16:57:55.297129 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:39249
--builtin_ntp_servers=127.8.14.84:36947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 16:57:55.297379 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:55.294962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:53545 (local address 127.8.14.126:39249)
0808 16:57:55.295008 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:55.295010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:55.295285 (+ 275us) server_negotiation.cc:407] Connection header received
0808 16:57:55.295790 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:55.295792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:55.295799 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:55.295849 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:55.297126 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:55.297135 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:55.297135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:57:55.297140 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:55.297155 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:57:55.297159 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:55.297211 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:55.297296 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 16:57:55.297310 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:57:55.297571 23399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:55.294977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:39249 (local address 127.8.14.65:53545)
0808 16:57:55.295162 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:55.295192 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:55.295662 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:55.295934 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:55.295954 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:55.295964 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:55.296974 (+ 1010us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:57:55.296974 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:55.297001 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:57:55.297028 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:55.297209 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:55.297214 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:55.297217 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:57:55.297293 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 16:57:55.297353 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260808 16:57:55.298151 23397 heartbeater.cc:347] Connected to a master server at 127.8.14.126:39249
I20260808 16:57:55.298260 23397 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:55.298408 23397 heartbeater.cc:511] Master 127.8.14.126:39249 requested a full tablet report, sending...
I20260808 16:57:55.298934 23090 ts_manager.cc:194] Registered new tserver with Master: 043fe230c2f14fcf9723f323a4f04c12 (127.8.14.65:37943)
W20260808 16:57:55.378441 23400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:55.378634 23400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:55.378656 23400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:55.379771 23400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:55.379805 23400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:55.379829 23400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:57:55.381101 23400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:36947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:39249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:55.381285 23400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:55.381532 23400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:55.386404 23409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:55.386420 23408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:55.386421 23412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:55.386965 23400 server_base.cc:1034] running on GCE node
I20260808 16:57:55.387184 23400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:55.387496 23400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:55.388654 23400 hybrid_clock.cc:648] HybridClock initialized: now 1786208275388648 us; error 32 us; skew 500 ppm
I20260808 16:57:55.389967 23400 webserver.cc:460] Webserver started at http://127.8.14.66:46631/ using document root <none> and password file <none>
I20260808 16:57:55.390242 23400 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:55.390292 23400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:55.390470 23400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:55.391564 23400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "c345c211f8364dd19b626036f354fff5"
format_stamp: "Formatted at 2026-08-08 16:57:55 on dist-test-slave-1nl6"
server_key: "ef69b541cf985331ea39482e229dddc7"
server_key_iv: "d25062d0ef7133149d0ddc8c25dd7566"
server_key_version: "encryptionkey@0"
I20260808 16:57:55.391927 23400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "c345c211f8364dd19b626036f354fff5"
format_stamp: "Formatted at 2026-08-08 16:57:55 on dist-test-slave-1nl6"
server_key: "ef69b541cf985331ea39482e229dddc7"
server_key_iv: "d25062d0ef7133149d0ddc8c25dd7566"
server_key_version: "encryptionkey@0"
I20260808 16:57:55.393247 23400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:57:55.393971 23422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:55.394150 23400 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:57:55.394258 23400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "c345c211f8364dd19b626036f354fff5"
format_stamp: "Formatted at 2026-08-08 16:57:55 on dist-test-slave-1nl6"
server_key: "ef69b541cf985331ea39482e229dddc7"
server_key_iv: "d25062d0ef7133149d0ddc8c25dd7566"
server_key_version: "encryptionkey@0"
I20260808 16:57:55.394333 23400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:55.397679 23400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:55.397940 23400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:55.398087 23400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:55.398300 23400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:55.398608 23400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:55.398670 23400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:55.398710 23400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:55.398738 23400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:55.404568 23400 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:39605
I20260808 16:57:55.404604 23608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:39605 every 8 connection(s)
I20260808 16:57:55.404953 23400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:57:55.407883 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:55.405669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:59223 (local address 127.8.14.126:39249)
0808 16:57:55.405735 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:55.405736 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:55.405975 (+ 239us) server_negotiation.cc:407] Connection header received
0808 16:57:55.406415 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:55.406416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:55.406420 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:55.406479 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:55.407694 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:55.407696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:55.407696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:57:55.407699 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:55.407710 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:57:55.407713 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:55.407757 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:55.407819 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 16:57:55.407832 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 16:57:55.408080 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:55.405641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:39249 (local address 127.8.14.66:59223)
0808 16:57:55.405845 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:55.405884 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:55.406331 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:55.406554 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:55.406558 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:55.406566 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:55.407562 (+ 996us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:57:55.407563 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:55.407590 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:57:55.407618 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:55.407756 (+ 138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:55.407758 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:55.407760 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:57:55.407809 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 16:57:55.407853 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":111,"threads_started":1}
I20260808 16:57:55.408639 23609 heartbeater.cc:347] Connected to a master server at 127.8.14.126:39249
I20260808 16:57:55.408742 23609 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:55.408900 23609 heartbeater.cc:511] Master 127.8.14.126:39249 requested a full tablet report, sending...
I20260808 16:57:55.409253 23090 ts_manager.cc:194] Registered new tserver with Master: c345c211f8364dd19b626036f354fff5 (127.8.14.66:39605)
I20260808 16:57:55.412680 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 23400
I20260808 16:57:55.412765 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:57:55.412971 8249 external_mini_cluster.cc:1428] Setting key c5439f6be5b2791bc013620408b7f7ed
I20260808 16:57:55.413873 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:39249
--builtin_ntp_servers=127.8.14.84:36947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 16:57:55.497426 23612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:55.497622 23612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:55.497646 23612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:55.498862 23612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:55.498966 23612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:55.499034 23612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:57:55.500416 23612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:36947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:39249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:55.500669 23612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:55.500973 23612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:55.506039 23622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:55.506047 23627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:55.506063 23621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:55.506326 23612 server_base.cc:1034] running on GCE node
I20260808 16:57:55.506520 23612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:55.506800 23612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:55.507978 23612 hybrid_clock.cc:648] HybridClock initialized: now 1786208275507974 us; error 52 us; skew 500 ppm
I20260808 16:57:55.509177 23612 webserver.cc:460] Webserver started at http://127.8.14.67:36301/ using document root <none> and password file <none>
I20260808 16:57:55.509435 23612 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:55.509507 23612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:55.509682 23612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:55.510696 23612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "44136cfc7dde45bbb65e993780d213e5"
format_stamp: "Formatted at 2026-08-08 16:57:55 on dist-test-slave-1nl6"
server_key: "ba56e8a63b348d934c85ecd177bcb1b7"
server_key_iv: "cd107a7c0ba6539248fcd4a2b2b4a129"
server_key_version: "encryptionkey@0"
I20260808 16:57:55.511088 23612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "44136cfc7dde45bbb65e993780d213e5"
format_stamp: "Formatted at 2026-08-08 16:57:55 on dist-test-slave-1nl6"
server_key: "ba56e8a63b348d934c85ecd177bcb1b7"
server_key_iv: "cd107a7c0ba6539248fcd4a2b2b4a129"
server_key_version: "encryptionkey@0"
I20260808 16:57:55.512364 23612 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:57:55.513058 23641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:55.513243 23612 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:57:55.513334 23612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "44136cfc7dde45bbb65e993780d213e5"
format_stamp: "Formatted at 2026-08-08 16:57:55 on dist-test-slave-1nl6"
server_key: "ba56e8a63b348d934c85ecd177bcb1b7"
server_key_iv: "cd107a7c0ba6539248fcd4a2b2b4a129"
server_key_version: "encryptionkey@0"
I20260808 16:57:55.513417 23612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:55.516755 23612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:55.517005 23612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:55.517146 23612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:55.517364 23612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:55.517673 23612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:55.517726 23612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:55.517764 23612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:55.517792 23612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:55.523408 23612 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:39993
I20260808 16:57:55.523478 23817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:39993 every 8 connection(s)
I20260808 16:57:55.523790 23612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 16:57:55.526548 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:55.524497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:54441 (local address 127.8.14.126:39249)
0808 16:57:55.524562 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:55.524564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:55.524754 (+ 190us) server_negotiation.cc:407] Connection header received
0808 16:57:55.525165 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:55.525166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:55.525172 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:55.525228 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:55.526365 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:55.526366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:55.526367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 16:57:55.526371 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:55.526385 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 16:57:55.526389 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:55.526439 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:55.526487 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 16:57:55.526498 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 16:57:55.526729 23821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:55.524455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:39249 (local address 127.8.14.67:54441)
0808 16:57:55.524632 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:55.524669 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:55.525079 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:55.525274 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:55.525277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:55.525284 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:55.526216 (+ 932us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 16:57:55.526217 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:55.526243 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 16:57:55.526270 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:55.526438 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:55.526440 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:55.526442 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:57:55.526493 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 16:57:55.526539 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260808 16:57:55.527196 23819 heartbeater.cc:347] Connected to a master server at 127.8.14.126:39249
I20260808 16:57:55.527285 23819 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:55.527392 23819 heartbeater.cc:511] Master 127.8.14.126:39249 requested a full tablet report, sending...
I20260808 16:57:55.527725 23090 ts_manager.cc:194] Registered new tserver with Master: 44136cfc7dde45bbb65e993780d213e5 (127.8.14.67:39993)
I20260808 16:57:55.529227 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 23612
I20260808 16:57:55.529306 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:57:55.529458 8249 external_mini_cluster.cc:1428] Setting key 907cc28c111ea7b966afc6fb5d969b9d
I20260808 16:57:55.530347 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 16:57:55.532227 23161 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:57:55.531707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33746 (local address 127.8.14.126:39249)
0808 16:57:55.531767 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:55.531769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:55.531925 (+ 156us) server_negotiation.cc:407] Connection header received
0808 16:57:55.531962 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:55.531963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:55.531967 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:55.532019 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:55.532165 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33746: BlockingRecv error: recv got EOF from 127.0.0.1:33746 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 16:57:55.532294 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 23192
I20260808 16:57:55.547567 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 23400
I20260808 16:57:55.565538 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 23612
I20260808 16:57:55.575718 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 23030
2026-08-08T16:57:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (577 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:57:55 dist-test-slave-1nl6 krb5kdc[23832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:57:55 dist-test-slave-1nl6 krb5kdc[23832](info): set up 2 sockets
Aug 08 16:57:55 dist-test-slave-1nl6 krb5kdc[23832](info): commencing operation
krb5kdc: starting...
W20260808 16:57:56.804061 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:57:56 dist-test-slave-1nl6 krb5kdc[23832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:57:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:57:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:57:56.923877 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:35075
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:41939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:35075
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:57.006518 23850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:57.006724 23850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:57.006747 23850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:57.007990 23850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:57:57.008033 23850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:57.008047 23850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:57.008060 23850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:57:57.008072 23850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:57:57.009531 23850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:35075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:35075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:57.009795 23850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:57.010023 23850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:57.015064 23865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:57.015157 23850 server_base.cc:1034] running on GCE node
W20260808 16:57:57.015071 23859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:57.015084 23868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:57.015517 23850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:57.015832 23850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:57.017002 23850 hybrid_clock.cc:648] HybridClock initialized: now 1786208277016999 us; error 40 us; skew 500 ppm
Aug 08 16:57:57 dist-test-slave-1nl6 krb5kdc[23832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208277, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:57.019476 23850 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:57:57.019963 23850 webserver.cc:460] Webserver started at http://127.8.14.126:43933/ using document root <none> and password file <none>
I20260808 16:57:57.020197 23850 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:57.020267 23850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:57.020424 23850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:57.021500 23850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "52886d22e697408aa809bb77c4faa53f"
format_stamp: "Formatted at 2026-08-08 16:57:57 on dist-test-slave-1nl6"
server_key: "e141d19a034b5c99b2c2106beb05d4cb"
server_key_iv: "e654dc9b1a814aa0af2a1c383fb051e5"
server_key_version: "encryptionkey@0"
I20260808 16:57:57.021886 23850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "52886d22e697408aa809bb77c4faa53f"
format_stamp: "Formatted at 2026-08-08 16:57:57 on dist-test-slave-1nl6"
server_key: "e141d19a034b5c99b2c2106beb05d4cb"
server_key_iv: "e654dc9b1a814aa0af2a1c383fb051e5"
server_key_version: "encryptionkey@0"
I20260808 16:57:57.023324 23850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:57:57.024169 23878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:57.024358 23850 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:57:57.024471 23850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "52886d22e697408aa809bb77c4faa53f"
format_stamp: "Formatted at 2026-08-08 16:57:57 on dist-test-slave-1nl6"
server_key: "e141d19a034b5c99b2c2106beb05d4cb"
server_key_iv: "e654dc9b1a814aa0af2a1c383fb051e5"
server_key_version: "encryptionkey@0"
I20260808 16:57:57.024549 23850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:57.039279 23850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:57.041255 23850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:57.041368 23850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:57.045830 23850 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:35075
I20260808 16:57:57.046049 23954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:35075 every 8 connection(s)
I20260808 16:57:57.046298 23850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:57:57.047349 23957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:57.048527 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 23850
I20260808 16:57:57.048645 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:57:57.048842 8249 external_mini_cluster.cc:1428] Setting key cb6bfbb0296176b398e83a41c12ffee1
I20260808 16:57:57.050267 23957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f: Bootstrap starting.
I20260808 16:57:57.050993 23957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:57.051347 23957 log.cc:824] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f: Log is configured to *not* fsync() on all Append() calls
I20260808 16:57:57.052098 23957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f: No bootstrap required, opened a new log
I20260808 16:57:57.054402 23957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52886d22e697408aa809bb77c4faa53f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35075 } }
I20260808 16:57:57.054580 23957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:57.054647 23957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52886d22e697408aa809bb77c4faa53f, State: Initialized, Role: FOLLOWER
I20260808 16:57:57.054781 23957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52886d22e697408aa809bb77c4faa53f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35075 } }
I20260808 16:57:57.054898 23957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:57:57.054967 23957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:57:57.055022 23957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:57.055660 23957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52886d22e697408aa809bb77c4faa53f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35075 } }
I20260808 16:57:57.055776 23957 leader_election.cc:302] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52886d22e697408aa809bb77c4faa53f; no voters:
I20260808 16:57:57.055948 23957 leader_election.cc:288] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:57:57.056007 23966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [term 1 FOLLOWER]: Leader election won for term 1
Aug 08 16:57:57 dist-test-slave-1nl6 krb5kdc[23832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:57:57.056166 23966 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [term 1 LEADER]: Becoming Leader. State: Replica: 52886d22e697408aa809bb77c4faa53f, State: Running, Role: LEADER
I20260808 16:57:57.056236 23957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:57:57.056246 23966 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52886d22e697408aa809bb77c4faa53f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35075 } }
I20260808 16:57:57.056598 23966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52886d22e697408aa809bb77c4faa53f. Latest consensus state: current_term: 1 leader_uuid: "52886d22e697408aa809bb77c4faa53f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52886d22e697408aa809bb77c4faa53f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35075 } } }
I20260808 16:57:57.056767 23966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [sys.catalog]: This master's current role is: LEADER
I20260808 16:57:57.057049 23967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52886d22e697408aa809bb77c4faa53f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52886d22e697408aa809bb77c4faa53f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35075 } } }
I20260808 16:57:57.057085 23976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:57:57.057232 23967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f [sys.catalog]: This master's current role is: LEADER
I20260808 16:57:57.057919 23976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:57:57.059767 23976 catalog_manager.cc:1347] Generated new cluster ID: acf9fbcd5b764933ab66ffef2f9293b5
I20260808 16:57:57.059820 23976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:57:57.060813 23961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.049381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51202 (local address 127.8.14.126:35075)
0808 16:57:57.049608 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:57.049618 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:57.049653 (+ 35us) server_negotiation.cc:407] Connection header received
0808 16:57:57.050213 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:57.050222 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:57.050282 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:57.050431 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:57.051092 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.052469 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.053180 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.053827 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.053959 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.056723 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:57.056727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:57.056727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:57.056739 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:57.059076 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:57.059506 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:57.059510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:57.059510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:57.059630 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:57.059883 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:57.059884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:57.059884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:57.060177 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:57.060315 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:57.060461 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 16:57:57.060607 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":116,"threads_started":1}
I20260808 16:57:57.067453 23976 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:57:57.068188 23976 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:57:57.074074 23976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f: Generated new TSK 0
I20260808 16:57:57.074241 23976 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 16:57:57.105327 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35075
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:41939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:57.198349 23994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:57.198557 23994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:57.198580 23994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:57.199810 23994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:57.199855 23994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:57.199882 23994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:57:57.201303 23994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:57.201541 23994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:57.201802 23994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:57:57.207801 23994 server_base.cc:1034] running on GCE node
W20260808 16:57:57.207814 24009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:57.207938 24011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:57.207983 24000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:57.208246 23994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:57.208508 23994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:57.209671 23994 hybrid_clock.cc:648] HybridClock initialized: now 1786208277209665 us; error 42 us; skew 500 ppm
Aug 08 16:57:57 dist-test-slave-1nl6 krb5kdc[23832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208277, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:57.212453 23994 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:57:57.212847 23994 webserver.cc:460] Webserver started at http://127.8.14.65:41539/ using document root <none> and password file <none>
I20260808 16:57:57.213027 23994 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:57.213065 23994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:57.213153 23994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:57.214143 23994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "869a2603ac8f41c6961073c255c3c559"
format_stamp: "Formatted at 2026-08-08 16:57:57 on dist-test-slave-1nl6"
server_key: "582fb72de525bc41520fd5b68ed4087d"
server_key_iv: "2edd360c1459fdb0b0c1e61cd9b7a283"
server_key_version: "encryptionkey@0"
I20260808 16:57:57.214447 23994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "869a2603ac8f41c6961073c255c3c559"
format_stamp: "Formatted at 2026-08-08 16:57:57 on dist-test-slave-1nl6"
server_key: "582fb72de525bc41520fd5b68ed4087d"
server_key_iv: "2edd360c1459fdb0b0c1e61cd9b7a283"
server_key_version: "encryptionkey@0"
I20260808 16:57:57.215865 23994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:57:57.216629 24022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:57.216850 23994 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:57:57.216950 23994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "869a2603ac8f41c6961073c255c3c559"
format_stamp: "Formatted at 2026-08-08 16:57:57 on dist-test-slave-1nl6"
server_key: "582fb72de525bc41520fd5b68ed4087d"
server_key_iv: "2edd360c1459fdb0b0c1e61cd9b7a283"
server_key_version: "encryptionkey@0"
I20260808 16:57:57.217027 23994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:57.228454 23994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:57.230726 23994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:57.230892 23994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:57.231148 23994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:57.231458 23994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:57.231492 23994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:57.231513 23994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:57.231562 23994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:57.237426 23994 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:40049
I20260808 16:57:57.237480 24188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:40049 every 8 connection(s)
I20260808 16:57:57.237789 23994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:57:57.240967 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 23994
I20260808 16:57:57.241079 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:57:57.241276 8249 external_mini_cluster.cc:1428] Setting key 72059d07cf0f966b7825ff9ca4fe2257
Aug 08 16:57:57 dist-test-slave-1nl6 krb5kdc[23832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208277, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:57:57.248600 23961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.238578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:49083 (local address 127.8.14.126:35075)
0808 16:57:57.238665 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:57.238667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:57.238749 (+ 82us) server_negotiation.cc:407] Connection header received
0808 16:57:57.239316 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:57.239317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:57.239327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:57.239382 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:57.240336 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.240850 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.242082 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.242294 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.242356 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.245268 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:57.245271 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:57.245271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:57.245280 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:57.247034 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:57.247445 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:57.247447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:57.247447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:57.247517 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:57.247770 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:57.247772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:57.247772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:57.247935 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:57.248019 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:57.248350 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 16:57:57.248456 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 16:57:57.248636 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.238488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:35075 (local address 127.8.14.65:49083)
0808 16:57:57.238645 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:57.238683 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:57.239219 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:57.239438 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:57.239442 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:57.240011 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:57.240226 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:57.240232 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.240953 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.240955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:57.241993 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:57.241997 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.242384 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.242385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:57.242527 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.243170 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:57.243181 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:57.245158 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:57.247115 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:57.247142 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:57.247144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:57.247364 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:57.247600 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:57.247600 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:57.247601 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:57.247700 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:57.248036 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:57.248044 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:57.248188 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 16:57:57.248295 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 16:57:57.248456 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
I20260808 16:57:57.249326 24191 heartbeater.cc:347] Connected to a master server at 127.8.14.126:35075
I20260808 16:57:57.249436 24191 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:57.249639 24191 heartbeater.cc:511] Master 127.8.14.126:35075 requested a full tablet report, sending...
I20260808 16:57:57.250206 23905 ts_manager.cc:194] Registered new tserver with Master: 869a2603ac8f41c6961073c255c3c559 (127.8.14.65:40049)
I20260808 16:57:57.250893 23905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:49083
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 16:57:57.270071 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35075
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:41939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:57.350889 24198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:57.351166 24198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:57.351207 24198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:57.352356 24198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:57.352422 24198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:57.352464 24198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:57:57.353916 24198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:57.354211 24198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:57.354451 24198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:57.359822 24208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:57.359865 24206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:57.359998 24198 server_base.cc:1034] running on GCE node
W20260808 16:57:57.359822 24205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:57.360458 24198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:57.360706 24198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:57.361858 24198 hybrid_clock.cc:648] HybridClock initialized: now 1786208277361846 us; error 29 us; skew 500 ppm
Aug 08 16:57:57 dist-test-slave-1nl6 krb5kdc[23832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208277, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:57.364334 24198 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:57:57.364811 24198 webserver.cc:460] Webserver started at http://127.8.14.66:39927/ using document root <none> and password file <none>
I20260808 16:57:57.365075 24198 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:57.365142 24198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:57.365293 24198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:57.366309 24198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "550baee6c93b4f0eaf560389c84abffe"
format_stamp: "Formatted at 2026-08-08 16:57:57 on dist-test-slave-1nl6"
server_key: "8d369399543e6a10576a22cca023ca55"
server_key_iv: "71350e1b4326f87b33ecc18f903d436d"
server_key_version: "encryptionkey@0"
I20260808 16:57:57.366689 24198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "550baee6c93b4f0eaf560389c84abffe"
format_stamp: "Formatted at 2026-08-08 16:57:57 on dist-test-slave-1nl6"
server_key: "8d369399543e6a10576a22cca023ca55"
server_key_iv: "71350e1b4326f87b33ecc18f903d436d"
server_key_version: "encryptionkey@0"
I20260808 16:57:57.368085 24198 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:57:57.368923 24216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:57.369136 24198 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 16:57:57.369230 24198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "550baee6c93b4f0eaf560389c84abffe"
format_stamp: "Formatted at 2026-08-08 16:57:57 on dist-test-slave-1nl6"
server_key: "8d369399543e6a10576a22cca023ca55"
server_key_iv: "71350e1b4326f87b33ecc18f903d436d"
server_key_version: "encryptionkey@0"
I20260808 16:57:57.369318 24198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:57.378662 24198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:57.380508 24198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:57.380671 24198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:57.380903 24198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:57.381201 24198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:57.381258 24198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:57.381299 24198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:57.381327 24198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:57.387724 24198 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:40489
I20260808 16:57:57.387768 24386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:40489 every 8 connection(s)
I20260808 16:57:57.388069 24198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
Aug 08 16:57:57 dist-test-slave-1nl6 krb5kdc[23832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208277, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:57:57.395301 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 24198
I20260808 16:57:57.395419 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:57:57.395671 8249 external_mini_cluster.cc:1428] Setting key a71cb9b37e14403a7d4008e68a09e07f
I20260808 16:57:57.397611 24389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.388699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:35075 (local address 127.8.14.66:52509)
0808 16:57:57.388853 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:57.388893 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:57.389356 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:57.389557 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:57.389560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:57.390103 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:57.390319 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:57.390324 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.391009 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.391009 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:57.391785 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:57.391787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.392205 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.392205 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:57.392301 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.392736 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:57.392744 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:57.394053 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:57.396156 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:57.396162 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:57.396164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:57.396485 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:57.396745 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:57.396746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:57.396746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:57.396849 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:57.397166 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:57.397170 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:57.397277 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 16:57:57.397354 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 16:57:57.397464 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260808 16:57:57.397668 23961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.388759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:52509 (local address 127.8.14.126:35075)
0808 16:57:57.388804 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:57.388807 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:57.388985 (+ 178us) server_negotiation.cc:407] Connection header received
0808 16:57:57.389440 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:57.389441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:57.389450 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:57.389508 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:57.390397 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.390877 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.391894 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.392094 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.392180 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.394204 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:57.394206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:57.394207 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:57.394214 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:57.396067 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:57.396568 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:57.396569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:57.396569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:57.396664 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:57.396932 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:57.396933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:57.396934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:57.397092 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:57.397165 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:57.397409 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 16:57:57.397507 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:57:57.398132 24387 heartbeater.cc:347] Connected to a master server at 127.8.14.126:35075
I20260808 16:57:57.398238 24387 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:57.398427 24387 heartbeater.cc:511] Master 127.8.14.126:35075 requested a full tablet report, sending...
I20260808 16:57:57.398968 23905 ts_manager.cc:194] Registered new tserver with Master: 550baee6c93b4f0eaf560389c84abffe (127.8.14.66:40489)
I20260808 16:57:57.399376 23905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:52509
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 16:57:57.423537 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35075
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:41939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:57:57.505748 24395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:57:57.505995 24395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:57:57.506035 24395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:57:57.507207 24395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:57:57.507283 24395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:57:57.507328 24395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:57:57.508883 24395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:57:57.509155 24395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:57:57.509404 24395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:57:57.514701 24413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:57.514695 24406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:57:57.514719 24409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:57:57.514861 24395 server_base.cc:1034] running on GCE node
I20260808 16:57:57.515182 24395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:57:57.515440 24395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:57:57.516613 24395 hybrid_clock.cc:648] HybridClock initialized: now 1786208277516606 us; error 35 us; skew 500 ppm
Aug 08 16:57:57 dist-test-slave-1nl6 krb5kdc[23832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208277, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:57:57.519191 24395 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:57:57.519620 24395 webserver.cc:460] Webserver started at http://127.8.14.67:44489/ using document root <none> and password file <none>
I20260808 16:57:57.519850 24395 fs_manager.cc:359] Metadata directory not provided
I20260808 16:57:57.519919 24395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:57:57.520107 24395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:57:57.521200 24395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "74b8f4ec5a2f465bb20efaa73b057198"
format_stamp: "Formatted at 2026-08-08 16:57:57 on dist-test-slave-1nl6"
server_key: "fa59ad963581df6b8a81df920cf82ce3"
server_key_iv: "57875688a30f3d101f0f667c36049e84"
server_key_version: "encryptionkey@0"
I20260808 16:57:57.521519 24395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "74b8f4ec5a2f465bb20efaa73b057198"
format_stamp: "Formatted at 2026-08-08 16:57:57 on dist-test-slave-1nl6"
server_key: "fa59ad963581df6b8a81df920cf82ce3"
server_key_iv: "57875688a30f3d101f0f667c36049e84"
server_key_version: "encryptionkey@0"
I20260808 16:57:57.522807 24395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:57:57.523617 24423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:57.523813 24395 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:57:57.523916 24395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "74b8f4ec5a2f465bb20efaa73b057198"
format_stamp: "Formatted at 2026-08-08 16:57:57 on dist-test-slave-1nl6"
server_key: "fa59ad963581df6b8a81df920cf82ce3"
server_key_iv: "57875688a30f3d101f0f667c36049e84"
server_key_version: "encryptionkey@0"
I20260808 16:57:57.523996 24395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:57:57.537582 24395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:57:57.539707 24395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:57:57.539881 24395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:57:57.540119 24395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:57:57.540444 24395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:57:57.540501 24395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:57.540535 24395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:57:57.540562 24395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:57:57.546639 24395 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:43067
I20260808 16:57:57.546689 24597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:43067 every 8 connection(s)
I20260808 16:57:57.547112 24395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 16:57:57.548599 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 24395
I20260808 16:57:57.548689 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:57:57.548938 8249 external_mini_cluster.cc:1428] Setting key d07387bc1fabf541a0abf5b826d206c9
Aug 08 16:57:57 dist-test-slave-1nl6 krb5kdc[23832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208277, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:57:57.556793 23961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.547876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:56317 (local address 127.8.14.126:35075)
0808 16:57:57.547924 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:57.547927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:57.548182 (+ 255us) server_negotiation.cc:407] Connection header received
0808 16:57:57.548767 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:57.548769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:57.548785 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:57.548857 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:57.550147 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.550646 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.551651 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.551852 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.551905 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.553999 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:57.554000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:57.554000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:57.554007 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:57.555328 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:57.555719 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:57.555720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:57.555720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:57.555772 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:57.556017 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:57.556018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:57.556019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:57.556188 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:57.556257 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:57.556660 (+ 403us) server_negotiation.cc:299] Negotiation successful
0808 16:57:57.556728 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 16:57:57.557020 24600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.547869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:35075 (local address 127.8.14.67:56317)
0808 16:57:57.548055 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:57.548089 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:57.548616 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:57.548955 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:57.548958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:57.549806 (+ 848us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:57:57.550070 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:57.550075 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.550729 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.550730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:57.551577 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:57.551579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.551933 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.551934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:57.552083 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.552559 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:57:57.552568 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:57:57.553911 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:57:57.555413 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:57.555419 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:57.555421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:57.555651 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:57.555831 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:57:57.555832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:57:57.555832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:57:57.555950 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:57:57.556355 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:57:57.556361 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:57:57.556520 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 16:57:57.556633 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 16:57:57.556765 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
I20260808 16:57:57.557664 24598 heartbeater.cc:347] Connected to a master server at 127.8.14.126:35075
I20260808 16:57:57.557782 24598 heartbeater.cc:464] Registering TS with master...
I20260808 16:57:57.557955 24598 heartbeater.cc:511] Master 127.8.14.126:35075 requested a full tablet report, sending...
I20260808 16:57:57.558360 23905 ts_manager.cc:194] Registered new tserver with Master: 74b8f4ec5a2f465bb20efaa73b057198 (127.8.14.67:43067)
I20260808 16:57:57.558708 23905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:56317
I20260808 16:57:57.560732 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:57:57.567921 23961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.562034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36554 (local address 127.8.14.126:35075)
0808 16:57:57.562084 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:57.562086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:57.562172 (+ 86us) server_negotiation.cc:407] Connection header received
0808 16:57:57.562213 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:57.562214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:57.562222 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:57.562264 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:57:57.562775 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.563299 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.564027 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.564215 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.564271 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.565501 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:57:57.565503 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:57:57.565503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:57:57.565509 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:57:57.566881 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:57.567249 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:57.567250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:57.567250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:57.567301 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:57:57.567506 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:57:57.567506 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:57:57.567507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:57:57.567634 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:57:57.567710 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:57.567807 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 16:57:57.567864 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 16:57:57.569392 23905 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:57:57.570955 23905 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:57:57.577662 24620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.573623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46868 (local address 127.8.14.66:40489)
0808 16:57:57.573784 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:57.573789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:57.573800 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:57:57.573842 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:57.573844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:57.573861 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:57.573945 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:57.574156 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.575363 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.576488 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.577230 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.577313 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.577378 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:57.577535 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 16:57:57.577574 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260808 16:57:57.577649 24614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.573264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:40049 (local address 127.0.0.1:49458)
0808 16:57:57.573448 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:57.573458 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:57.573499 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:57.574026 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:57.574027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:57.574031 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:57.574085 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:57.574087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.575185 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.575187 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:57.576473 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:57.576476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.577344 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.577345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:57.577438 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.577441 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:57:57.577475 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 16:57:57.577550 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":10,"thread_start_us":55,"threads_started":1}
I20260808 16:57:57.577661 24613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.573351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:43067 (local address 127.0.0.1:45796)
0808 16:57:57.573581 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:57.573593 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:57.573650 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:57.573972 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:57.573975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:57.573978 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:57.574051 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:57.574053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.575520 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.575521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:57.576711 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:57.576714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.577393 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.577393 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:57.577441 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.577444 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:57:57.577480 (+ 36us) client_negotiation.cc:240] Negotiation successful
0808 16:57:57.577542 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":29,"thread_start_us":49,"threads_started":1}
I20260808 16:57:57.577968 24616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.573599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45796 (local address 127.8.14.67:43067)
0808 16:57:57.573782 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:57.573787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:57.573798 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:57:57.573838 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:57.573840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:57.573853 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:57.574221 (+ 368us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:57.574304 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.575366 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.576800 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.577266 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.577705 (+ 439us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.577798 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:57.577829 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 16:57:57.577886 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":117,"threads_started":1}
I20260808 16:57:57.578097 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.573413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49458 (local address 127.8.14.65:40049)
0808 16:57:57.573857 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:57.573861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:57.573870 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:57:57.573908 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:57.573910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:57.573925 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:57.574164 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:57.574237 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.575086 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.576577 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.577244 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.577844 (+ 600us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.577919 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:57.577952 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 16:57:57.577986 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":79,"threads_started":1}
I20260808 16:57:57.577661 24612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.573140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:40489 (local address 127.0.0.1:46868)
0808 16:57:57.573276 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:57.573327 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:57.573418 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:57.574026 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:57.574027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:57.574029 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:57.574086 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:57.574087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.575485 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.575486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:57.576401 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:57.576403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.577328 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.577329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:57.577417 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.577432 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 16:57:57.577507 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 16:57:57.577544 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":2,"thread_start_us":60,"threads_started":1}
I20260808 16:57:57.578845 24098 tablet_service.cc:1467] Processing CreateTablet for tablet ef312ae1c79946bf80a198ea0363bd71 (DEFAULT_TABLE table=test-table [id=da4cab3c1f4a487481166862b0791164]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:57.578843 24497 tablet_service.cc:1467] Processing CreateTablet for tablet ef312ae1c79946bf80a198ea0363bd71 (DEFAULT_TABLE table=test-table [id=da4cab3c1f4a487481166862b0791164]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:57.579274 24098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef312ae1c79946bf80a198ea0363bd71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:57.579280 24497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef312ae1c79946bf80a198ea0363bd71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:57.581739 24621 tablet_bootstrap.cc:492] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198: Bootstrap starting.
I20260808 16:57:57.582095 24622 tablet_bootstrap.cc:492] T ef312ae1c79946bf80a198ea0363bd71 P 869a2603ac8f41c6961073c255c3c559: Bootstrap starting.
I20260808 16:57:57.582145 24290 tablet_service.cc:1467] Processing CreateTablet for tablet ef312ae1c79946bf80a198ea0363bd71 (DEFAULT_TABLE table=test-table [id=da4cab3c1f4a487481166862b0791164]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:57:57.582494 24290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef312ae1c79946bf80a198ea0363bd71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:57:57.582515 24621 tablet_bootstrap.cc:654] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:57.582772 24622 tablet_bootstrap.cc:654] T ef312ae1c79946bf80a198ea0363bd71 P 869a2603ac8f41c6961073c255c3c559: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:57.583138 24621 log.cc:824] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198: Log is configured to *not* fsync() on all Append() calls
I20260808 16:57:57.583364 24622 log.cc:824] T ef312ae1c79946bf80a198ea0363bd71 P 869a2603ac8f41c6961073c255c3c559: Log is configured to *not* fsync() on all Append() calls
I20260808 16:57:57.583900 24621 tablet_bootstrap.cc:492] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198: No bootstrap required, opened a new log
I20260808 16:57:57.583982 24621 ts_tablet_manager.cc:1397] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:57:57.584193 24622 tablet_bootstrap.cc:492] T ef312ae1c79946bf80a198ea0363bd71 P 869a2603ac8f41c6961073c255c3c559: No bootstrap required, opened a new log
I20260808 16:57:57.584262 24622 ts_tablet_manager.cc:1397] T ef312ae1c79946bf80a198ea0363bd71 P 869a2603ac8f41c6961073c255c3c559: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 16:57:57.585520 24627 tablet_bootstrap.cc:492] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe: Bootstrap starting.
I20260808 16:57:57.586127 24627 tablet_bootstrap.cc:654] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe: Neither blocks nor log segments found. Creating new log.
I20260808 16:57:57.586268 24621 raft_consensus.cc:348] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550baee6c93b4f0eaf560389c84abffe" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40489 } } peers { permanent_uuid: "869a2603ac8f41c6961073c255c3c559" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40049 } } peers { permanent_uuid: "74b8f4ec5a2f465bb20efaa73b057198" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 43067 } }
I20260808 16:57:57.586241 24622 raft_consensus.cc:348] T ef312ae1c79946bf80a198ea0363bd71 P 869a2603ac8f41c6961073c255c3c559 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550baee6c93b4f0eaf560389c84abffe" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40489 } } peers { permanent_uuid: "869a2603ac8f41c6961073c255c3c559" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40049 } } peers { permanent_uuid: "74b8f4ec5a2f465bb20efaa73b057198" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 43067 } }
I20260808 16:57:57.586419 24621 raft_consensus.cc:374] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:57.586421 24622 raft_consensus.cc:374] T ef312ae1c79946bf80a198ea0363bd71 P 869a2603ac8f41c6961073c255c3c559 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:57.586457 24622 raft_consensus.cc:729] T ef312ae1c79946bf80a198ea0363bd71 P 869a2603ac8f41c6961073c255c3c559 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 869a2603ac8f41c6961073c255c3c559, State: Initialized, Role: FOLLOWER
I20260808 16:57:57.586457 24621 raft_consensus.cc:729] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74b8f4ec5a2f465bb20efaa73b057198, State: Initialized, Role: FOLLOWER
I20260808 16:57:57.586599 24621 consensus_queue.cc:260] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550baee6c93b4f0eaf560389c84abffe" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40489 } } peers { permanent_uuid: "869a2603ac8f41c6961073c255c3c559" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40049 } } peers { permanent_uuid: "74b8f4ec5a2f465bb20efaa73b057198" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 43067 } }
I20260808 16:57:57.586599 24622 consensus_queue.cc:260] T ef312ae1c79946bf80a198ea0363bd71 P 869a2603ac8f41c6961073c255c3c559 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550baee6c93b4f0eaf560389c84abffe" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40489 } } peers { permanent_uuid: "869a2603ac8f41c6961073c255c3c559" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40049 } } peers { permanent_uuid: "74b8f4ec5a2f465bb20efaa73b057198" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 43067 } }
I20260808 16:57:57.586696 24627 log.cc:824] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe: Log is configured to *not* fsync() on all Append() calls
I20260808 16:57:57.586877 24622 ts_tablet_manager.cc:1428] T ef312ae1c79946bf80a198ea0363bd71 P 869a2603ac8f41c6961073c255c3c559: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 16:57:57.586877 24621 ts_tablet_manager.cc:1428] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:57:57.587026 24191 heartbeater.cc:503] Master 127.8.14.126:35075 was elected leader, sending a full tablet report...
I20260808 16:57:57.587111 24598 heartbeater.cc:503] Master 127.8.14.126:35075 was elected leader, sending a full tablet report...
I20260808 16:57:57.587503 24627 tablet_bootstrap.cc:492] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe: No bootstrap required, opened a new log
I20260808 16:57:57.587577 24627 ts_tablet_manager.cc:1397] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 16:57:57.589164 24627 raft_consensus.cc:348] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550baee6c93b4f0eaf560389c84abffe" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40489 } } peers { permanent_uuid: "869a2603ac8f41c6961073c255c3c559" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40049 } } peers { permanent_uuid: "74b8f4ec5a2f465bb20efaa73b057198" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 43067 } }
I20260808 16:57:57.589299 24627 raft_consensus.cc:374] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:57:57.589322 24627 raft_consensus.cc:729] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 550baee6c93b4f0eaf560389c84abffe, State: Initialized, Role: FOLLOWER
I20260808 16:57:57.589449 24627 consensus_queue.cc:260] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550baee6c93b4f0eaf560389c84abffe" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40489 } } peers { permanent_uuid: "869a2603ac8f41c6961073c255c3c559" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40049 } } peers { permanent_uuid: "74b8f4ec5a2f465bb20efaa73b057198" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 43067 } }
I20260808 16:57:57.589709 24627 ts_tablet_manager.cc:1428] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 16:57:57.589802 24387 heartbeater.cc:503] Master 127.8.14.126:35075 was elected leader, sending a full tablet report...
W20260808 16:57:57.638594 24388 tablet.cc:2369] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:57:57.721014 24635 raft_consensus.cc:482] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:57:57.721138 24635 raft_consensus.cc:504] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550baee6c93b4f0eaf560389c84abffe" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40489 } } peers { permanent_uuid: "869a2603ac8f41c6961073c255c3c559" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40049 } } peers { permanent_uuid: "74b8f4ec5a2f465bb20efaa73b057198" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 43067 } }
I20260808 16:57:57.721565 24635 leader_election.cc:288] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 869a2603ac8f41c6961073c255c3c559 (127.8.14.65:40049), 74b8f4ec5a2f465bb20efaa73b057198 (127.8.14.67:43067)
I20260808 16:57:57.725835 24389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.721772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:43067 (local address 127.8.14.66:56991)
0808 16:57:57.721877 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:57.721907 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:57.721961 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:57.722166 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:57.722167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:57.722168 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:57.722260 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:57.722263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.723335 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.723336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:57.724665 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:57.724669 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.725554 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.725555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:57.725646 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.725648 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 16:57:57.725704 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 16:57:57.725746 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":86}
I20260808 16:57:57.725847 24616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.721864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:56991 (local address 127.8.14.67:43067)
0808 16:57:57.721941 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:57.721943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:57.721954 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:57:57.722032 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:57.722034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:57.722054 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:57.722150 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:57.722373 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.723229 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.724781 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.725456 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.725547 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.725559 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:57.725731 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 16:57:57.725772 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 16:57:57.725847 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.721945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:58877 (local address 127.8.14.65:40049)
0808 16:57:57.722005 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:57.722007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:57.722096 (+ 89us) server_negotiation.cc:407] Connection header received
0808 16:57:57.722134 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:57.722135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:57.722147 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:57.722208 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:57:57.722366 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.723241 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.724781 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.725464 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.725547 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.725558 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:57.725731 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 16:57:57.725772 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 16:57:57.725883 24638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.721798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:40049 (local address 127.8.14.66:58877)
0808 16:57:57.722013 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:57.722023 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:57:57.722079 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:57:57.722222 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:57:57.722223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:57:57.722225 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:57:57.722282 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:57.722284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.723335 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.723336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:57.724665 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:57:57.724669 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.725554 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.725555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:57:57.725638 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.725642 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:57:57.725704 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 16:57:57.725746 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":40,"thread_start_us":140,"threads_started":1}
I20260808 16:57:57.726243 24525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef312ae1c79946bf80a198ea0363bd71" candidate_uuid: "550baee6c93b4f0eaf560389c84abffe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74b8f4ec5a2f465bb20efaa73b057198" is_pre_election: true
I20260808 16:57:57.726344 24123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef312ae1c79946bf80a198ea0363bd71" candidate_uuid: "550baee6c93b4f0eaf560389c84abffe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "869a2603ac8f41c6961073c255c3c559" is_pre_election: true
I20260808 16:57:57.726434 24525 raft_consensus.cc:2454] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 550baee6c93b4f0eaf560389c84abffe in term 0.
I20260808 16:57:57.726507 24123 raft_consensus.cc:2454] T ef312ae1c79946bf80a198ea0363bd71 P 869a2603ac8f41c6961073c255c3c559 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 550baee6c93b4f0eaf560389c84abffe in term 0.
I20260808 16:57:57.726745 24221 leader_election.cc:302] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 550baee6c93b4f0eaf560389c84abffe, 74b8f4ec5a2f465bb20efaa73b057198; no voters:
I20260808 16:57:57.726941 24635 raft_consensus.cc:2781] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:57:57.727056 24635 raft_consensus.cc:482] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:57:57.727087 24635 raft_consensus.cc:3037] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:57.727867 24635 raft_consensus.cc:504] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550baee6c93b4f0eaf560389c84abffe" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40489 } } peers { permanent_uuid: "869a2603ac8f41c6961073c255c3c559" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40049 } } peers { permanent_uuid: "74b8f4ec5a2f465bb20efaa73b057198" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 43067 } }
I20260808 16:57:57.728017 24635 leader_election.cc:288] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [CANDIDATE]: Term 1 election: Requested vote from peers 869a2603ac8f41c6961073c255c3c559 (127.8.14.65:40049), 74b8f4ec5a2f465bb20efaa73b057198 (127.8.14.67:43067)
I20260808 16:57:57.728237 24123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef312ae1c79946bf80a198ea0363bd71" candidate_uuid: "550baee6c93b4f0eaf560389c84abffe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "869a2603ac8f41c6961073c255c3c559"
I20260808 16:57:57.728267 24525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef312ae1c79946bf80a198ea0363bd71" candidate_uuid: "550baee6c93b4f0eaf560389c84abffe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74b8f4ec5a2f465bb20efaa73b057198"
I20260808 16:57:57.728335 24525 raft_consensus.cc:3037] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:57.728340 24123 raft_consensus.cc:3037] T ef312ae1c79946bf80a198ea0363bd71 P 869a2603ac8f41c6961073c255c3c559 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:57:57.729048 24525 raft_consensus.cc:2454] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 550baee6c93b4f0eaf560389c84abffe in term 1.
I20260808 16:57:57.729147 24123 raft_consensus.cc:2454] T ef312ae1c79946bf80a198ea0363bd71 P 869a2603ac8f41c6961073c255c3c559 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 550baee6c93b4f0eaf560389c84abffe in term 1.
I20260808 16:57:57.729211 24221 leader_election.cc:302] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 550baee6c93b4f0eaf560389c84abffe, 74b8f4ec5a2f465bb20efaa73b057198; no voters:
I20260808 16:57:57.729343 24635 raft_consensus.cc:2781] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:57:57.729486 24635 raft_consensus.cc:686] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [term 1 LEADER]: Becoming Leader. State: Replica: 550baee6c93b4f0eaf560389c84abffe, State: Running, Role: LEADER
I20260808 16:57:57.729591 24635 consensus_queue.cc:237] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550baee6c93b4f0eaf560389c84abffe" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40489 } } peers { permanent_uuid: "869a2603ac8f41c6961073c255c3c559" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40049 } } peers { permanent_uuid: "74b8f4ec5a2f465bb20efaa73b057198" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 43067 } }
I20260808 16:57:57.730456 23903 catalog_manager.cc:5495] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe reported cstate change: term changed from 0 to 1, leader changed from <none> to 550baee6c93b4f0eaf560389c84abffe (127.8.14.66). New cstate: current_term: 1 leader_uuid: "550baee6c93b4f0eaf560389c84abffe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "550baee6c93b4f0eaf560389c84abffe" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40489 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "869a2603ac8f41c6961073c255c3c559" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74b8f4ec5a2f465bb20efaa73b057198" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 43067 } health_report { overall_health: UNKNOWN } } }
W20260808 16:57:57.738457 24192 tablet.cc:2369] T ef312ae1c79946bf80a198ea0363bd71 P 869a2603ac8f41c6961073c255c3c559: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:57:57.758780 24620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.755899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46876 (local address 127.8.14.66:40489)
0808 16:57:57.755968 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 16:57:57.755971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:57:57.755981 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:57:57.756063 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:57:57.756065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:57:57.756094 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:57:57.756172 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:57:57.756358 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.757026 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.757849 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:57:57.758120 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:57:57.758211 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:57:57.758408 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:57:57.758526 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:57:57.758618 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 16:57:57.758664 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 16:57:57.758685 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 16:57:57.760725 24123 raft_consensus.cc:1261] T ef312ae1c79946bf80a198ea0363bd71 P 869a2603ac8f41c6961073c255c3c559 [term 1 FOLLOWER]: Refusing update from remote peer 550baee6c93b4f0eaf560389c84abffe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:57:57.761047 24525 raft_consensus.cc:1261] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198 [term 1 FOLLOWER]: Refusing update from remote peer 550baee6c93b4f0eaf560389c84abffe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:57:57.761111 24635 consensus_queue.cc:1035] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [LEADER]: Connected to new peer: Peer: permanent_uuid: "869a2603ac8f41c6961073c255c3c559" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 40049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:57:57.761318 24635 consensus_queue.cc:1035] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [LEADER]: Connected to new peer: Peer: permanent_uuid: "74b8f4ec5a2f465bb20efaa73b057198" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 43067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:57:57.763550 24646 mvcc.cc:205] Tried to move back new op lower bound from 7316309105704841216 to 7316309105581068288. Current Snapshot: MvccSnapshot[applied={T|T < 7316309105704841216}]
I20260808 16:57:57.763880 24650 mvcc.cc:205] Tried to move back new op lower bound from 7316309105704841216 to 7316309105581068288. Current Snapshot: MvccSnapshot[applied={T|T < 7316309105704841216}]
I20260808 16:57:57.763984 24647 mvcc.cc:205] Tried to move back new op lower bound from 7316309105704841216 to 7316309105581068288. Current Snapshot: MvccSnapshot[applied={T|T < 7316309105704841216}]
I20260808 16:57:57.766201 23903 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:57:57.766305 23903 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:57:57.767112 23903 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52886d22e697408aa809bb77c4faa53f: Sending DeleteTablet for 3 replicas of tablet ef312ae1c79946bf80a198ea0363bd71
I20260808 16:57:57.767431 24098 tablet_service.cc:1514] Processing DeleteTablet for tablet ef312ae1c79946bf80a198ea0363bd71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:57 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:49458
I20260808 16:57:57.767488 24497 tablet_service.cc:1514] Processing DeleteTablet for tablet ef312ae1c79946bf80a198ea0363bd71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:57 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:45796
I20260808 16:57:57.767719 24290 tablet_service.cc:1514] Processing DeleteTablet for tablet ef312ae1c79946bf80a198ea0363bd71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:57:57 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:46868
I20260808 16:57:57.767845 24662 tablet_replica.cc:330] T ef312ae1c79946bf80a198ea0363bd71 P 869a2603ac8f41c6961073c255c3c559: stopping tablet replica
I20260808 16:57:57.767881 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 23994
I20260808 16:57:57.768450 24663 tablet_replica.cc:330] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198: stopping tablet replica
I20260808 16:57:57.771682 24663 raft_consensus.cc:2229] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:57:57.771858 24663 raft_consensus.cc:2258] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:57.773299 24663 ts_tablet_manager.cc:1905] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:57.775125 24663 ts_tablet_manager.cc:1918] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:57:57.775285 24663 log.cc:1196] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/ef312ae1c79946bf80a198ea0363bd71
I20260808 16:57:57.775596 24663 ts_tablet_manager.cc:1939] T ef312ae1c79946bf80a198ea0363bd71 P 74b8f4ec5a2f465bb20efaa73b057198: Deleting consensus metadata
I20260808 16:57:57.776034 23883 catalog_manager.cc:4841] TS 74b8f4ec5a2f465bb20efaa73b057198 (127.8.14.67:43067): tablet ef312ae1c79946bf80a198ea0363bd71 (table test-table [id=da4cab3c1f4a487481166862b0791164]) successfully deleted
I20260808 16:57:57.778446 24665 tablet_replica.cc:330] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe: stopping tablet replica
I20260808 16:57:57.778586 24665 raft_consensus.cc:2229] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:57:57.778720 24665 raft_consensus.cc:2258] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:57:57.779176 24665 ts_tablet_manager.cc:1905] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:57:57.780471 24665 ts_tablet_manager.cc:1918] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:57:57.780567 24665 log.cc:1196] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/ef312ae1c79946bf80a198ea0363bd71
I20260808 16:57:57.780831 24665 ts_tablet_manager.cc:1939] T ef312ae1c79946bf80a198ea0363bd71 P 550baee6c93b4f0eaf560389c84abffe: Deleting consensus metadata
I20260808 16:57:57.781879 23883 catalog_manager.cc:4841] TS 550baee6c93b4f0eaf560389c84abffe (127.8.14.66:40489): tablet ef312ae1c79946bf80a198ea0363bd71 (table test-table [id=da4cab3c1f4a487481166862b0791164]) successfully deleted
W20260808 16:57:57.786983 23882 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.14.65:40049 (error 108)
I20260808 16:57:57.787494 24612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:57:57.787364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:40049 (local address 127.0.0.1:49472)
0808 16:57:57.787420 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 16:57:57.787466 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.65:40049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 16:57:57.787567 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 24198
W20260808 16:57:57.787698 23882 catalog_manager.cc:4584] TS 869a2603ac8f41c6961073c255c3c559 (127.8.14.65:40049): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ef312ae1c79946bf80a198ea0363bd71: Network error: Client connection negotiation failed: client connection to 127.8.14.65:40049: connect: Connection refused (error 111)
I20260808 16:57:57.796639 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 24395
I20260808 16:57:57.802861 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 23850
2026-08-08T16:57:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2221 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:57:57 dist-test-slave-1nl6 krb5kdc[24671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:57:57 dist-test-slave-1nl6 krb5kdc[24671](info): set up 2 sockets
Aug 08 16:57:57 dist-test-slave-1nl6 krb5kdc[24671](info): commencing operation
krb5kdc: starting...
W20260808 16:57:59.048256 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:57:59 dist-test-slave-1nl6 krb5kdc[24671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:57:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:57:59Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.14.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.14.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.14.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/rangeradmin_127.8.14.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.14.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/rangeradmin_127.8.14.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.14.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.14.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.14.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/rangerlookup_127.8.14.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.14.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/rangerlookup_127.8.14.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.14.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.14.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.14.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/HTTP_127.8.14.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.14.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/HTTP_127.8.14.84@KRBTEST.COM.keytab.
I20260808 16:57:59.213280 8249 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskiOuv3j/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/postgres -l logfile start
2026-08-08 16:58:01.551 UTC [24719] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 16:58:01.552 UTC [24719] LOG: listening on IPv4 address "127.8.14.84", port 55195
2026-08-08 16:58:01.555 UTC [24719] LOG: listening on Unix socket "/tmp/.s.PGSQL.55195"
2026-08-08 16:58:01.589 UTC [24722] LOG: database system was shut down at 2026-08-08 16:58:00 UTC
2026-08-08 16:58:01.593 UTC [24719] LOG: database system is ready to accept connections
I20260808 16:58:02.718142 8249 mini_postgres.cc:96] Postgres bound to 55195
2026-08-08 16:58:02.726 UTC [24735] FATAL: database "slave" does not exist
127.8.14.84:55195 - accepting connections
I20260808 16:58:02.726812 8249 mini_ranger.cc:161] Starting Ranger...
I20260808 16:58:02.742420 8249 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 16:58:03.159883 8249 mini_ranger.cc:87] Created ranger Postgres database
I20260808 16:58:03.159971 8249 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ranger-admin
2026-08-08 16:58:03,530 [I] DB FLAVOR :POSTGRES
2026-08-08 16:58:03,530 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 16:58:03,530 [I] Checking connection..
2026-08-08 16:58:03,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 16:58:03,829 [I] Checking connection passed.
2026-08-08 16:58:03,830 [I] --------- Verifying version history table ---------
2026-08-08 16:58:03,830 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 16:58:04,223 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 16:58:04,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 16:58:04,589 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 16:58:04,589 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 16:58:04,589 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 16:58:04.965 UTC [24839] WARNING: there is no transaction in progress
2026-08-08 16:58:04,979 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 16:58:04,979 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 16:58:05,309 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 16:58:05,309 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 16:58:05,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 16:58:05,546 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 16:58:05,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1nl6.c.gcp-upstream.internal','N') ;"
2026-08-08 16:58:06,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 16:58:06,313 [I] Table x_portal_user does not exist in database ranger
2026-08-08 16:58:06,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 16:58:06,558 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 16:58:06,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 16:58:06,791 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 16:58:07,053 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 16:58:07,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 16:58:07.973 UTC [25000] WARNING: there is no transaction in progress
2026-08-08 16:58:07.989 UTC [25000] WARNING: there is no transaction in progress
2026-08-08 16:58:08.004 UTC [25000] WARNING: there is no transaction in progress
2026-08-08 16:58:08.018 UTC [25000] WARNING: there is no transaction in progress
2026-08-08 16:58:08.032 UTC [25000] WARNING: there is no transaction in progress
2026-08-08 16:58:08.048 UTC [25000] WARNING: there is no transaction in progress
2026-08-08 16:58:08.098 UTC [25000] WARNING: there is no transaction in progress
2026-08-08 16:58:08.104 UTC [25000] WARNING: there is no transaction in progress
2026-08-08 16:58:08.117 UTC [25000] WARNING: there is no transaction in progress
2026-08-08 16:58:08.126 UTC [25000] WARNING: there is no transaction in progress
2026-08-08 16:58:08.136 UTC [25000] WARNING: there is no transaction in progress
2026-08-08 16:58:08.150 UTC [25000] WARNING: there is no transaction in progress
2026-08-08 16:58:08.159 UTC [25000] WARNING: there is no transaction in progress
2026-08-08 16:58:08.164 UTC [25000] WARNING: there is no transaction in progress
2026-08-08 16:58:08.170 UTC [25000] WARNING: there is no transaction in progress
2026-08-08 16:58:08.176 UTC [25000] WARNING: there is no transaction in progress
2026-08-08 16:58:08,750 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 16:58:08,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 16:58:08,997 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 16:58:08,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 16:58:09,253 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 16:58:09,253 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 16:58:09,501 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 16:58:09,502 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 16:58:09,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 16:58:10,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1nl6.c.gcp-upstream.internal';"
2026-08-08 16:58:10,260 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 16:58:10,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 16:58:10,511 [I] DB_PATCHES have already been applied
I20260808 16:58:10.516880 8249 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ranger-admin:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskiOuv3j/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 16:58:10.516947 8249 mini_ranger.cc:193] Using host: 127.8.14.84
I20260808 16:58:10.521031 8249 mini_ranger.cc:227] Ranger admin URL: http://127.8.14.84:38723
Aug 08, 2026 4:58:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 4:58:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 4:58:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 4:58:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 4:58:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 4:58:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 4:58:11 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 16:58:11.854797 8249 mini_ranger.cc:160] Time spent starting Ranger: real 9.128s user 0.001s sys 0.010s
Aug 08, 2026 4:58:32 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 4:58:32 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 4:58:32 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 4:58:32 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 4:58:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 4:58:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 4:58:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 4:58:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 4:58:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 4:58:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 4:58:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 4:58:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 4:58:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 4:58:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 4:58:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 4:58:32 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 4:58:32 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 4:58:33 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 16:58:34.147020 8249 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.14.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.14.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.14.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/rangerkms_127.8.14.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.14.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/rangerkms_127.8.14.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.14.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.14.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.14.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/HTTP_127.8.14.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.14.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/HTTP_127.8.14.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 16:58:34 dist-test-slave-1nl6 krb5kdc[24671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208314, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 16:58:34.247248 8249 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 16:58:34.259058 8249 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 16:58:35.031679 8249 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 16:58:35.031771 8249 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ranger-kms
I20260808 16:58:35.031793 8249 mini_ranger_kms.cc:227] Using postgres at 127.8.14.84:55195
2026-08-08 16:58:35,589 [I] DB FLAVOR :POSTGRES
2026-08-08 16:58:35,589 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 16:58:35,589 [I] Checking connection
2026-08-08 16:58:35,589 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 16:58:35,825 [I] connection success
2026-08-08 16:58:35,825 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 16:58:35,825 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 16:58:35,825 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 16:58:36,067 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 16:58:36,067 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 16:58:36,068 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 16:58:36,068 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:55195/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 16:58:36,351 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 16:58:36.823835 8249 mini_ranger_kms.cc:310] Created kms service
I20260808 16:58:37.023869 8249 mini_ranger_kms.cc:326] Created kudu user
I20260808 16:58:37.069190 8249 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 16:58:37 dist-test-slave-1nl6 krb5kdc[24671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208314, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:58:37.461323 8249 mini_ranger_kms.cc:384] Added ranger policy
I20260808 16:58:37.461445 8249 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ranger-kms:/tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 16:58:37.461467 8249 mini_ranger_kms.cc:240] Using host: 127.8.14.84
I20260808 16:58:37.465672 8249 mini_ranger_kms.cc:276] Ranger KMS PID: 25264
I20260808 16:58:37.465804 8249 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.14.84:39879
Aug 08, 2026 4:58:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 4:58:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 4:58:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 4:58:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 4:58:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 4:58:39 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 16:58:39 dist-test-slave-1nl6 krb5kdc[24671](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786208319, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.14.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:58:40.132784 8249 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.886s user 0.011s sys 0.222s
I20260808 16:58:40.132912 8249 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 16:58:40.132998 8249 mini_ranger_kms.cc:401] 127.8.14.84:39879/kms/v1/keys
Aug 08 16:58:40 dist-test-slave-1nl6 krb5kdc[24671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208319, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.14.84@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
Aug 08 16:58:40 dist-test-slave-1nl6 krb5kdc[24671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208319, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.14.84@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
Aug 08 16:58:42 dist-test-slave-1nl6 krb5kdc[24671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:58:43.009580 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:42377
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:35657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:42377
--ranger_config_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.14.84:39879/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:58:43.108294 25339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:58:43.108528 25339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:58:43.108564 25339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:58:43.109704 25339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:58:43.109771 25339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:58:43.109794 25339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:58:43.109827 25339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:58:43.109854 25339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:58:43.111575 25339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:35657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.14.84:39879/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:42377
--ranger_config_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:42377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:58:43.111863 25339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:58:43.112097 25339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:58:43.117424 25339 server_base.cc:1034] running on GCE node
W20260808 16:58:43.117437 25350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:58:43.117473 25348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:58:43.117610 25352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:58:43.117945 25339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:58:43.118280 25339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:58:43.119468 25339 hybrid_clock.cc:648] HybridClock initialized: now 1786208323119443 us; error 43 us; skew 500 ppm
Aug 08 16:58:43 dist-test-slave-1nl6 krb5kdc[24671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208323, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:58:43.122406 25339 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:58:43.122973 25339 webserver.cc:460] Webserver started at http://127.8.14.126:46293/ using document root <none> and password file <none>
I20260808 16:58:43.123209 25339 fs_manager.cc:359] Metadata directory not provided
I20260808 16:58:43.123250 25339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:58:43.123389 25339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 16:58:43 dist-test-slave-1nl6 krb5kdc[24671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208323, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:58:43.184738 25339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "ccb4d0c1c75e4efbafb6d303cc60d22b"
format_stamp: "Formatted at 2026-08-08 16:58:43 on dist-test-slave-1nl6"
server_key: "522738fe351dee8e9f38ff1fe6e909b8"
server_key_iv: "40e8994b09237e36dd7ca16f9f41447b"
server_key_version: "kuduclusterkey@0"
I20260808 16:58:43.185102 25339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "ccb4d0c1c75e4efbafb6d303cc60d22b"
format_stamp: "Formatted at 2026-08-08 16:58:43 on dist-test-slave-1nl6"
server_key: "522738fe351dee8e9f38ff1fe6e909b8"
server_key_iv: "40e8994b09237e36dd7ca16f9f41447b"
server_key_version: "kuduclusterkey@0"
I20260808 16:58:43.186800 25339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 16:58:43.201282 25363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:58:43.201514 25339 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:58:43.201625 25339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "ccb4d0c1c75e4efbafb6d303cc60d22b"
format_stamp: "Formatted at 2026-08-08 16:58:43 on dist-test-slave-1nl6"
server_key: "522738fe351dee8e9f38ff1fe6e909b8"
server_key_iv: "40e8994b09237e36dd7ca16f9f41447b"
server_key_version: "kuduclusterkey@0"
I20260808 16:58:43.201743 25339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:58:43.209401 25339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:58:43.211474 25339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:58:43.211644 25339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:58:43.216766 25339 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:42377
I20260808 16:58:43.216813 25450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:42377 every 8 connection(s)
I20260808 16:58:43.217163 25339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:58:43.217901 25453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:58:43.220376 25453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b: Bootstrap starting.
I20260808 16:58:43.221081 25453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b: Neither blocks nor log segments found. Creating new log.
I20260808 16:58:43.221380 25453 log.cc:824] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b: Log is configured to *not* fsync() on all Append() calls
I20260808 16:58:43.222110 25453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b: No bootstrap required, opened a new log
I20260808 16:58:43.224334 25453 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccb4d0c1c75e4efbafb6d303cc60d22b" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42377 } }
I20260808 16:58:43.224486 25453 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:58:43.224511 25453 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb4d0c1c75e4efbafb6d303cc60d22b, State: Initialized, Role: FOLLOWER
I20260808 16:58:43.224664 25453 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccb4d0c1c75e4efbafb6d303cc60d22b" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42377 } }
I20260808 16:58:43.224740 25453 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:58:43.224761 25453 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:58:43.224829 25453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:58:43.225354 25453 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccb4d0c1c75e4efbafb6d303cc60d22b" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42377 } }
I20260808 16:58:43.225464 25453 leader_election.cc:302] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccb4d0c1c75e4efbafb6d303cc60d22b; no voters:
I20260808 16:58:43.225666 25453 leader_election.cc:288] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:58:43.225719 25458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:58:43.225929 25458 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [term 1 LEADER]: Becoming Leader. State: Replica: ccb4d0c1c75e4efbafb6d303cc60d22b, State: Running, Role: LEADER
I20260808 16:58:43.225960 25453 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:58:43.226023 25458 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccb4d0c1c75e4efbafb6d303cc60d22b" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42377 } }
I20260808 16:58:43.226392 25458 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccb4d0c1c75e4efbafb6d303cc60d22b. Latest consensus state: current_term: 1 leader_uuid: "ccb4d0c1c75e4efbafb6d303cc60d22b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccb4d0c1c75e4efbafb6d303cc60d22b" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42377 } } }
I20260808 16:58:43.226403 25459 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccb4d0c1c75e4efbafb6d303cc60d22b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccb4d0c1c75e4efbafb6d303cc60d22b" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42377 } } }
I20260808 16:58:43.226495 25458 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [sys.catalog]: This master's current role is: LEADER
I20260808 16:58:43.226516 25459 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b [sys.catalog]: This master's current role is: LEADER
I20260808 16:58:43.226766 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 25339
I20260808 16:58:43.226861 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:58:43.227080 8249 external_mini_cluster.cc:1428] Setting key 780d12d41f37c4a4b512d535ccc32392
I20260808 16:58:43.228180 25453 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 16:58:43.231129 25466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:58:43.232314 25466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:58:43.234566 25466 catalog_manager.cc:1347] Generated new cluster ID: 5ea2fd33596b47d086dcb022e2d726fd
I20260808 16:58:43.234620 25466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:58:43 dist-test-slave-1nl6 krb5kdc[24671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:58:43.261632 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:43.227541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41312 (local address 127.8.14.126:42377)
0808 16:58:43.239018 (+ 11477us) server_negotiation.cc:206] Beginning negotiation
0808 16:58:43.239031 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 16:58:43.239073 (+ 42us) server_negotiation.cc:407] Connection header received
0808 16:58:43.239212 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:58:43.239219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:58:43.239275 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:58:43.239444 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:58:43.250979 (+ 11535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:43.252506 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:43.253662 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:43.254308 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:43.254440 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:43.257328 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:58:43.257332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:58:43.257332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:58:43.257345 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:58:43.259928 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:58:43.260340 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:58:43.260343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:58:43.260343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:58:43.260461 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:58:43.260680 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:58:43.260680 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:58:43.260681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:58:43.261003 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:58:43.261157 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 16:58:43.261273 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 16:58:43.261398 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11410,"thread_start_us":77,"threads_started":1}
I20260808 16:58:43.283667 25466 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:58:43.284499 25466 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:58:43.291024 25466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b: Generated new TSK 0
I20260808 16:58:43.291280 25466 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 16:58:43.975854 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.14.84:39879/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:42377
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:35657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:58:44.101809 25519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:58:44.102118 25519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:58:44.102159 25519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:58:44.103888 25519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:58:44.103948 25519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:58:44.103991 25519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:58:44.106176 25519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:35657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.14.84:39879/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:42377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:58:44.106446 25519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:58:44.106729 25519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:58:44.112265 25528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:58:44.112437 25532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:58:44.112310 25526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:58:44.112557 25519 server_base.cc:1034] running on GCE node
I20260808 16:58:44.112775 25519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:58:44.113055 25519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:58:44.114240 25519 hybrid_clock.cc:648] HybridClock initialized: now 1786208324114214 us; error 43 us; skew 500 ppm
Aug 08 16:58:44 dist-test-slave-1nl6 krb5kdc[24671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208324, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:58:44.121426 25519 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:58:44.121994 25519 webserver.cc:460] Webserver started at http://127.8.14.65:43349/ using document root <none> and password file <none>
I20260808 16:58:44.122260 25519 fs_manager.cc:359] Metadata directory not provided
I20260808 16:58:44.122334 25519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:58:44.122485 25519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 16:58:44 dist-test-slave-1nl6 krb5kdc[24671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208324, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:58:44.142192 25519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "ca3113b8417d4a9b867af7236bfd57ea"
format_stamp: "Formatted at 2026-08-08 16:58:44 on dist-test-slave-1nl6"
server_key: "3c03037662f2511f246e0bc9dad16d61"
server_key_iv: "da39563889b7058fd4aebbb430516405"
server_key_version: "kuduclusterkey@0"
I20260808 16:58:44.142695 25519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "ca3113b8417d4a9b867af7236bfd57ea"
format_stamp: "Formatted at 2026-08-08 16:58:44 on dist-test-slave-1nl6"
server_key: "3c03037662f2511f246e0bc9dad16d61"
server_key_iv: "da39563889b7058fd4aebbb430516405"
server_key_version: "kuduclusterkey@0"
I20260808 16:58:44.144493 25519 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 16:58:44.160579 25547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:58:44.160877 25519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 16:58:44.161003 25519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "ca3113b8417d4a9b867af7236bfd57ea"
format_stamp: "Formatted at 2026-08-08 16:58:44 on dist-test-slave-1nl6"
server_key: "3c03037662f2511f246e0bc9dad16d61"
server_key_iv: "da39563889b7058fd4aebbb430516405"
server_key_version: "kuduclusterkey@0"
I20260808 16:58:44.161104 25519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:58:44.174152 25519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:58:44.179015 25519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:58:44.179231 25519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:58:44.179538 25519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:58:44.179934 25519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:58:44.179982 25519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:58:44.180022 25519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:58:44.180040 25519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:58:44.187777 25519 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:36623
I20260808 16:58:44.188272 25519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:58:44.187852 25705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:36623 every 8 connection(s)
I20260808 16:58:44.194416 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 25519
I20260808 16:58:44.194535 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:58:44.194746 8249 external_mini_cluster.cc:1428] Setting key 1629295c48d87b350e4421e3f0fb474b
Aug 08 16:58:44 dist-test-slave-1nl6 krb5kdc[24671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208324, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:58:44.198808 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.189537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42377 (local address 127.8.14.65:49169)
0808 16:58:44.189720 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 16:58:44.189763 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 16:58:44.190311 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:58:44.190597 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:58:44.190606 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:58:44.190793 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:58:44.191056 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:58:44.191062 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.191776 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.191777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:58:44.192580 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:58:44.192582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.193007 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.193008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:58:44.193131 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.193672 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:58:44.193681 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:58:44.195158 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:58:44.197208 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:58:44.197215 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:58:44.197217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:58:44.197508 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:58:44.197724 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:58:44.197725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:58:44.197725 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:58:44.197855 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:58:44.198138 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:58:44.198144 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:58:44.198314 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 16:58:44.198422 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 16:58:44.198570 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260808 16:58:44.199536 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.189554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:49169 (local address 127.8.14.126:42377)
0808 16:58:44.189938 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0808 16:58:44.189941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:58:44.189952 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:58:44.190429 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:58:44.190430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:58:44.190446 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:58:44.190525 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:58:44.191163 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.191677 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.192677 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.192906 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.192994 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.195269 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:58:44.195271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:58:44.195271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:58:44.195279 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:58:44.197111 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:58:44.197594 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:58:44.197596 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:58:44.197596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:58:44.197659 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:58:44.197925 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:58:44.197926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:58:44.197927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:58:44.198059 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:58:44.198113 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 16:58:44.199406 (+ 1293us) server_negotiation.cc:299] Negotiation successful
0808 16:58:44.199474 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":87,"threads_started":1}
I20260808 16:58:44.199934 25707 heartbeater.cc:347] Connected to a master server at 127.8.14.126:42377
I20260808 16:58:44.200088 25707 heartbeater.cc:464] Registering TS with master...
I20260808 16:58:44.200284 25707 heartbeater.cc:511] Master 127.8.14.126:42377 requested a full tablet report, sending...
I20260808 16:58:44.200822 25397 ts_manager.cc:194] Registered new tserver with Master: ca3113b8417d4a9b867af7236bfd57ea (127.8.14.65:36623)
I20260808 16:58:44.201442 25397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:49169
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 16:58:44.225965 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.14.84:39879/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:42377
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:35657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:58:44.345376 25721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:58:44.345712 25721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:58:44.345804 25721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:58:44.347503 25721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:58:44.347569 25721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:58:44.347610 25721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:58:44.349727 25721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:35657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.14.84:39879/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:42377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:58:44.350191 25721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:58:44.350489 25721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:58:44.356827 25741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:58:44.356841 25733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:58:44.357173 25736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:58:44.357743 25721 server_base.cc:1034] running on GCE node
I20260808 16:58:44.357995 25721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:58:44.358284 25721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Aug 08 16:58:44 dist-test-slave-1nl6 krb5kdc[24671](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786208324, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:58:44.359475 25721 hybrid_clock.cc:648] HybridClock initialized: now 1786208324359453 us; error 34 us; skew 500 ppm
Aug 08 16:58:44 dist-test-slave-1nl6 krb5kdc[24671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208324, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:58:44.362775 25721 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:58:44.363337 25721 webserver.cc:460] Webserver started at http://127.8.14.66:37953/ using document root <none> and password file <none>
I20260808 16:58:44.363595 25721 fs_manager.cc:359] Metadata directory not provided
I20260808 16:58:44.363720 25721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:58:44.363903 25721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 16:58:44 dist-test-slave-1nl6 krb5kdc[24671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208324, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:58:44.393631 25721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "12eb854662c24e21be8e0436e177cbf8"
format_stamp: "Formatted at 2026-08-08 16:58:44 on dist-test-slave-1nl6"
server_key: "ee45666a61c773603e8817a71f89fffb"
server_key_iv: "e12b221eba2038b1487303cf346b8adc"
server_key_version: "kuduclusterkey@0"
I20260808 16:58:44.394184 25721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "12eb854662c24e21be8e0436e177cbf8"
format_stamp: "Formatted at 2026-08-08 16:58:44 on dist-test-slave-1nl6"
server_key: "ee45666a61c773603e8817a71f89fffb"
server_key_iv: "e12b221eba2038b1487303cf346b8adc"
server_key_version: "kuduclusterkey@0"
I20260808 16:58:44.395756 25721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 16:58:44.410166 25754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:58:44.410409 25721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:58:44.410517 25721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "12eb854662c24e21be8e0436e177cbf8"
format_stamp: "Formatted at 2026-08-08 16:58:44 on dist-test-slave-1nl6"
server_key: "ee45666a61c773603e8817a71f89fffb"
server_key_iv: "e12b221eba2038b1487303cf346b8adc"
server_key_version: "kuduclusterkey@0"
I20260808 16:58:44.410640 25721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:58:44.444429 25721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:58:44.446887 25721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:58:44.447088 25721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:58:44.447355 25721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:58:44.447746 25721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:58:44.447803 25721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:58:44.447844 25721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:58:44.447871 25721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:58:44.455222 25721 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:43023
I20260808 16:58:44.455294 25914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:43023 every 8 connection(s)
I20260808 16:58:44.455718 25721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:58:44.461154 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 25721
I20260808 16:58:44.461273 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:58:44.461511 8249 external_mini_cluster.cc:1428] Setting key c46f4c404bed594a14a23d8d35a3d5d1
Aug 08 16:58:44 dist-test-slave-1nl6 krb5kdc[24671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208324, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:58:44.467115 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.456773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:49501 (local address 127.8.14.126:42377)
0808 16:58:44.456837 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 16:58:44.456839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:58:44.456853 (+ 14us) server_negotiation.cc:407] Connection header received
0808 16:58:44.457470 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:58:44.457471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:58:44.457484 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:58:44.457557 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:58:44.458282 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.459129 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.460643 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.460969 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.461015 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.463729 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:58:44.463731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:58:44.463731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:58:44.463739 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:58:44.465481 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:58:44.466017 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:58:44.466018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:58:44.466019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:58:44.466077 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:58:44.466347 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:58:44.466348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:58:44.466348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:58:44.466477 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:58:44.466518 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 16:58:44.466921 (+ 403us) server_negotiation.cc:299] Negotiation successful
0808 16:58:44.467015 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 16:58:44.467842 25917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.456464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42377 (local address 127.8.14.66:49501)
0808 16:58:44.456672 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 16:58:44.456720 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 16:58:44.457337 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:58:44.457633 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:58:44.457637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:58:44.457885 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:58:44.458156 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:58:44.458163 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.459255 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.459256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:58:44.460522 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:58:44.460525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.461108 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.461109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:58:44.461250 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.461965 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:58:44.461975 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:58:44.463601 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:58:44.465631 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:58:44.465637 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:58:44.465640 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:58:44.465924 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:58:44.466149 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:58:44.466150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:58:44.466150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:58:44.466272 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:58:44.466565 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:58:44.466572 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:58:44.466748 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 16:58:44.467446 (+ 698us) client_negotiation.cc:240] Negotiation successful
0808 16:58:44.467605 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":100,"threads_started":1}
I20260808 16:58:44.468541 25915 heartbeater.cc:347] Connected to a master server at 127.8.14.126:42377
I20260808 16:58:44.468643 25915 heartbeater.cc:464] Registering TS with master...
I20260808 16:58:44.468820 25915 heartbeater.cc:511] Master 127.8.14.126:42377 requested a full tablet report, sending...
I20260808 16:58:44.469244 25397 ts_manager.cc:194] Registered new tserver with Master: 12eb854662c24e21be8e0436e177cbf8 (127.8.14.66:43023)
I20260808 16:58:44.469741 25397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:49501
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 16:58:44.495805 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.14.84:39879/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:42377
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:35657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:58:44.619547 25925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:58:44.619784 25925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:58:44.619820 25925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:58:44.620946 25925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:58:44.621014 25925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:58:44.621059 25925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:58:44.622524 25925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:35657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.14.84:39879/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:42377
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:58:44.622782 25925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:58:44.623065 25925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:58:44.629184 25933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:58:44.629231 25935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:58:44.629412 25937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:58:44.629670 25925 server_base.cc:1034] running on GCE node
I20260808 16:58:44.629873 25925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:58:44.630139 25925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:58:44.631304 25925 hybrid_clock.cc:648] HybridClock initialized: now 1786208324631285 us; error 31 us; skew 500 ppm
Aug 08 16:58:44 dist-test-slave-1nl6 krb5kdc[24671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208324, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:58:44.634212 25925 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:58:44.634859 25925 webserver.cc:460] Webserver started at http://127.8.14.67:35635/ using document root <none> and password file <none>
I20260808 16:58:44.635198 25925 fs_manager.cc:359] Metadata directory not provided
I20260808 16:58:44.635284 25925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:58:44.635457 25925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 16:58:44 dist-test-slave-1nl6 krb5kdc[24671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208324, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:58:44.649185 25925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "deac2568d92f4b5695be7fbd429e7269"
format_stamp: "Formatted at 2026-08-08 16:58:44 on dist-test-slave-1nl6"
server_key: "74170de3801755d08c9fcc8696365c0b"
server_key_iv: "5c219212d2f1dd589c72f404913a1d7d"
server_key_version: "kuduclusterkey@0"
I20260808 16:58:44.649698 25925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "deac2568d92f4b5695be7fbd429e7269"
format_stamp: "Formatted at 2026-08-08 16:58:44 on dist-test-slave-1nl6"
server_key: "74170de3801755d08c9fcc8696365c0b"
server_key_iv: "5c219212d2f1dd589c72f404913a1d7d"
server_key_version: "kuduclusterkey@0"
I20260808 16:58:44.655383 25925 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260808 16:58:44.673581 25949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:58:44.673786 25925 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 16:58:44.673887 25925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "deac2568d92f4b5695be7fbd429e7269"
format_stamp: "Formatted at 2026-08-08 16:58:44 on dist-test-slave-1nl6"
server_key: "74170de3801755d08c9fcc8696365c0b"
server_key_iv: "5c219212d2f1dd589c72f404913a1d7d"
server_key_version: "kuduclusterkey@0"
I20260808 16:58:44.673980 25925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:58:44.691059 25925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:58:44.693565 25925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:58:44.693739 25925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:58:44.693964 25925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:58:44.694370 25925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:58:44.694424 25925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:58:44.694460 25925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:58:44.694484 25925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:58:44.701675 25925 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:36885
I20260808 16:58:44.702131 25925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 16:58:44.705927 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 25925
I20260808 16:58:44.706044 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:58:44.706264 8249 external_mini_cluster.cc:1428] Setting key 5e3d27c9aa3d7ffaa6b5e6acbc1c7621
I20260808 16:58:44.707458 26132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:36885 every 8 connection(s)
Aug 08 16:58:44 dist-test-slave-1nl6 krb5kdc[24671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208324, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:58:44.717195 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.707918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:48093 (local address 127.8.14.126:42377)
0808 16:58:44.707964 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:58:44.707966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:58:44.708244 (+ 278us) server_negotiation.cc:407] Connection header received
0808 16:58:44.708777 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:58:44.708779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:58:44.708804 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:58:44.708838 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:58:44.709675 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.710427 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.711493 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.711698 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.711776 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.714108 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:58:44.714109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:58:44.714110 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:58:44.714118 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:58:44.715642 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:58:44.716164 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:58:44.716165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:58:44.716166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:58:44.716228 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:58:44.716506 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:58:44.716507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:58:44.716507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:58:44.716625 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:58:44.716699 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 16:58:44.717012 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 16:58:44.717109 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:58:44.717516 26136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.707932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:42377 (local address 127.8.14.67:48093)
0808 16:58:44.708096 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 16:58:44.708138 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 16:58:44.708633 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:58:44.709033 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:58:44.709038 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:58:44.709289 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:58:44.709563 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:58:44.709569 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.710522 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.710523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:58:44.711392 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:58:44.711394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.711781 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.711782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:58:44.711911 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.712523 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:58:44.712532 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:58:44.714010 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:58:44.715748 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:58:44.715755 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:58:44.715757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:58:44.716079 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:58:44.716317 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:58:44.716318 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:58:44.716318 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:58:44.716436 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:58:44.716703 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:58:44.716710 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:58:44.716866 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 16:58:44.716973 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 16:58:44.717130 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260808 16:58:44.717939 26134 heartbeater.cc:347] Connected to a master server at 127.8.14.126:42377
I20260808 16:58:44.718101 26134 heartbeater.cc:464] Registering TS with master...
I20260808 16:58:44.718361 26134 heartbeater.cc:511] Master 127.8.14.126:42377 requested a full tablet report, sending...
I20260808 16:58:44.718808 25397 ts_manager.cc:194] Registered new tserver with Master: deac2568d92f4b5695be7fbd429e7269 (127.8.14.67:36885)
I20260808 16:58:44.719201 25397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:48093
I20260808 16:58:44.728995 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:58:44.737747 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.731061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41318 (local address 127.8.14.126:42377)
0808 16:58:44.731114 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 16:58:44.731117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:58:44.731393 (+ 276us) server_negotiation.cc:407] Connection header received
0808 16:58:44.731458 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:58:44.731459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:58:44.731471 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:58:44.731515 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:58:44.732099 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.732590 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.733335 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.733510 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.733580 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.735062 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:58:44.735064 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:58:44.735064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:58:44.735071 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:58:44.736610 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:58:44.736937 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:58:44.736938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:58:44.736939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:58:44.737000 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:58:44.737204 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:58:44.737205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:58:44.737205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:58:44.737324 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:58:44.737429 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 16:58:44.737569 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 16:58:44.737642 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 16:58:44.739575 25397 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:58:44.741812 25397 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:58:44.750211 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.744710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:43023 (local address 127.0.0.1:41044)
0808 16:58:44.745726 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0808 16:58:44.745733 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 16:58:44.745793 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:58:44.746076 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:58:44.746079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:58:44.746081 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:58:44.746158 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:58:44.746160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.747673 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.747674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:58:44.748944 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:58:44.748947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.749943 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.749945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:58:44.750024 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.750040 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 16:58:44.750084 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 16:58:44.750131 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1006,"thread_start_us":83,"threads_started":1}
I20260808 16:58:44.750820 26161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.745035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41044 (local address 127.8.14.66:43023)
0808 16:58:44.745180 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 16:58:44.745185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:58:44.745808 (+ 623us) server_negotiation.cc:407] Connection header received
0808 16:58:44.745868 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:58:44.745870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:58:44.745887 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:58:44.745968 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:58:44.746256 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.747588 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.749066 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.749846 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.750550 (+ 704us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.750643 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 16:58:44.750671 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 16:58:44.750711 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260808 16:58:44.752404 25817 tablet_service.cc:1467] Processing CreateTablet for tablet 0dd517abcdb5498b8da058f07e65cbae (DEFAULT_TABLE table=test-table [id=2689be47e92b44d58c40fb84904f4af2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:58:44.752820 25817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dd517abcdb5498b8da058f07e65cbae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:58:44.754242 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.744471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:36623 (local address 127.0.0.1:60696)
0808 16:58:44.745208 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0808 16:58:44.745248 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 16:58:44.745368 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:58:44.746314 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:58:44.746314 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:58:44.746316 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:58:44.746348 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:58:44.746349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.750983 (+ 4634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.750983 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:58:44.751705 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:58:44.751708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.754016 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.754017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:58:44.754095 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.754100 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:58:44.754131 (+ 31us) client_negotiation.cc:240] Negotiation successful
0808 16:58:44.754174 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":84,"threads_started":1}
I20260808 16:58:44.754660 26158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.744736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60696 (local address 127.8.14.65:36623)
0808 16:58:44.745898 (+ 1162us) server_negotiation.cc:206] Beginning negotiation
0808 16:58:44.745904 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 16:58:44.745916 (+ 12us) server_negotiation.cc:407] Connection header received
0808 16:58:44.745949 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:58:44.745950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:58:44.745966 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:58:44.746549 (+ 583us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:58:44.746612 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.748299 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.751987 (+ 3688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.752733 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.754444 (+ 1711us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.754514 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 16:58:44.754542 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 16:58:44.754586 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1129,"thread_start_us":72,"threads_started":1}
I20260808 16:58:44.755201 26154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.744471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:36885 (local address 127.0.0.1:34348)
0808 16:58:44.745605 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0808 16:58:44.745614 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:58:44.745656 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:58:44.746424 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:58:44.746424 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:58:44.746426 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:58:44.746468 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:58:44.746469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.749173 (+ 2704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.749174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:58:44.750437 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:58:44.750439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.754987 (+ 4548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.754988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:58:44.755063 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.755068 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:58:44.755101 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 16:58:44.755136 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1117,"mutex_wait_us":5,"thread_start_us":84,"threads_started":1}
I20260808 16:58:44.755614 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.744931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34348 (local address 127.8.14.67:36885)
0808 16:58:44.746114 (+ 1183us) server_negotiation.cc:206] Beginning negotiation
0808 16:58:44.746117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:58:44.746126 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:58:44.746162 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:58:44.746164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:58:44.746180 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:58:44.746970 (+ 790us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:58:44.747045 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.749081 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.750884 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.753901 (+ 3017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.755381 (+ 1480us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.755467 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 16:58:44.755495 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 16:58:44.755537 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1146,"thread_start_us":86,"threads_started":1}
I20260808 16:58:44.756279 26016 tablet_service.cc:1467] Processing CreateTablet for tablet 0dd517abcdb5498b8da058f07e65cbae (DEFAULT_TABLE table=test-table [id=2689be47e92b44d58c40fb84904f4af2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:58:44.756276 25617 tablet_service.cc:1467] Processing CreateTablet for tablet 0dd517abcdb5498b8da058f07e65cbae (DEFAULT_TABLE table=test-table [id=2689be47e92b44d58c40fb84904f4af2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:58:44.756597 25617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dd517abcdb5498b8da058f07e65cbae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:58:44.756598 26016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dd517abcdb5498b8da058f07e65cbae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:58:44.760438 26163 tablet_bootstrap.cc:492] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269: Bootstrap starting.
I20260808 16:58:44.761504 26163 tablet_bootstrap.cc:654] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269: Neither blocks nor log segments found. Creating new log.
I20260808 16:58:44.761845 26163 log.cc:824] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269: Log is configured to *not* fsync() on all Append() calls
I20260808 16:58:44.762084 26165 tablet_bootstrap.cc:492] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8: Bootstrap starting.
I20260808 16:58:44.761902 26162 tablet_bootstrap.cc:492] T 0dd517abcdb5498b8da058f07e65cbae P ca3113b8417d4a9b867af7236bfd57ea: Bootstrap starting.
I20260808 16:58:44.762653 26163 tablet_bootstrap.cc:492] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269: No bootstrap required, opened a new log
I20260808 16:58:44.762737 26163 ts_tablet_manager.cc:1397] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:58:44.763974 26162 tablet_bootstrap.cc:654] T 0dd517abcdb5498b8da058f07e65cbae P ca3113b8417d4a9b867af7236bfd57ea: Neither blocks nor log segments found. Creating new log.
I20260808 16:58:44.764348 26162 log.cc:824] T 0dd517abcdb5498b8da058f07e65cbae P ca3113b8417d4a9b867af7236bfd57ea: Log is configured to *not* fsync() on all Append() calls
I20260808 16:58:44.764866 26165 tablet_bootstrap.cc:654] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8: Neither blocks nor log segments found. Creating new log.
I20260808 16:58:44.765465 26165 log.cc:824] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8: Log is configured to *not* fsync() on all Append() calls
I20260808 16:58:44.766371 26162 tablet_bootstrap.cc:492] T 0dd517abcdb5498b8da058f07e65cbae P ca3113b8417d4a9b867af7236bfd57ea: No bootstrap required, opened a new log
I20260808 16:58:44.766513 26162 ts_tablet_manager.cc:1397] T 0dd517abcdb5498b8da058f07e65cbae P ca3113b8417d4a9b867af7236bfd57ea: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 16:58:44.768913 26163 raft_consensus.cc:348] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deac2568d92f4b5695be7fbd429e7269" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 36885 } } peers { permanent_uuid: "12eb854662c24e21be8e0436e177cbf8" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 43023 } } peers { permanent_uuid: "ca3113b8417d4a9b867af7236bfd57ea" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 36623 } }
I20260808 16:58:44.769112 26163 raft_consensus.cc:374] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:58:44.769160 26163 raft_consensus.cc:729] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: deac2568d92f4b5695be7fbd429e7269, State: Initialized, Role: FOLLOWER
I20260808 16:58:44.769269 26163 consensus_queue.cc:260] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deac2568d92f4b5695be7fbd429e7269" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 36885 } } peers { permanent_uuid: "12eb854662c24e21be8e0436e177cbf8" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 43023 } } peers { permanent_uuid: "ca3113b8417d4a9b867af7236bfd57ea" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 36623 } }
I20260808 16:58:44.769554 26163 ts_tablet_manager.cc:1428] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260808 16:58:44.769523 26162 raft_consensus.cc:348] T 0dd517abcdb5498b8da058f07e65cbae P ca3113b8417d4a9b867af7236bfd57ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deac2568d92f4b5695be7fbd429e7269" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 36885 } } peers { permanent_uuid: "12eb854662c24e21be8e0436e177cbf8" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 43023 } } peers { permanent_uuid: "ca3113b8417d4a9b867af7236bfd57ea" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 36623 } }
I20260808 16:58:44.769699 26134 heartbeater.cc:503] Master 127.8.14.126:42377 was elected leader, sending a full tablet report...
I20260808 16:58:44.769779 26162 raft_consensus.cc:374] T 0dd517abcdb5498b8da058f07e65cbae P ca3113b8417d4a9b867af7236bfd57ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:58:44.769881 26162 raft_consensus.cc:729] T 0dd517abcdb5498b8da058f07e65cbae P ca3113b8417d4a9b867af7236bfd57ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca3113b8417d4a9b867af7236bfd57ea, State: Initialized, Role: FOLLOWER
I20260808 16:58:44.770066 26162 consensus_queue.cc:260] T 0dd517abcdb5498b8da058f07e65cbae P ca3113b8417d4a9b867af7236bfd57ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deac2568d92f4b5695be7fbd429e7269" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 36885 } } peers { permanent_uuid: "12eb854662c24e21be8e0436e177cbf8" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 43023 } } peers { permanent_uuid: "ca3113b8417d4a9b867af7236bfd57ea" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 36623 } }
I20260808 16:58:44.770381 26162 ts_tablet_manager.cc:1428] T 0dd517abcdb5498b8da058f07e65cbae P ca3113b8417d4a9b867af7236bfd57ea: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 16:58:44.770537 25707 heartbeater.cc:503] Master 127.8.14.126:42377 was elected leader, sending a full tablet report...
I20260808 16:58:44.771358 26165 tablet_bootstrap.cc:492] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8: No bootstrap required, opened a new log
I20260808 16:58:44.771445 26165 ts_tablet_manager.cc:1397] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260808 16:58:44.773104 26165 raft_consensus.cc:348] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deac2568d92f4b5695be7fbd429e7269" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 36885 } } peers { permanent_uuid: "12eb854662c24e21be8e0436e177cbf8" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 43023 } } peers { permanent_uuid: "ca3113b8417d4a9b867af7236bfd57ea" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 36623 } }
I20260808 16:58:44.773278 26165 raft_consensus.cc:374] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:58:44.773319 26165 raft_consensus.cc:729] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12eb854662c24e21be8e0436e177cbf8, State: Initialized, Role: FOLLOWER
I20260808 16:58:44.773473 26165 consensus_queue.cc:260] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deac2568d92f4b5695be7fbd429e7269" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 36885 } } peers { permanent_uuid: "12eb854662c24e21be8e0436e177cbf8" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 43023 } } peers { permanent_uuid: "ca3113b8417d4a9b867af7236bfd57ea" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 36623 } }
I20260808 16:58:44.773754 26165 ts_tablet_manager.cc:1428] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:58:44.773806 25915 heartbeater.cc:503] Master 127.8.14.126:42377 was elected leader, sending a full tablet report...
I20260808 16:58:44.810747 26173 raft_consensus.cc:482] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:58:44.810904 26173 raft_consensus.cc:504] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deac2568d92f4b5695be7fbd429e7269" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 36885 } } peers { permanent_uuid: "12eb854662c24e21be8e0436e177cbf8" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 43023 } } peers { permanent_uuid: "ca3113b8417d4a9b867af7236bfd57ea" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 36623 } }
I20260808 16:58:44.811795 26173 leader_election.cc:288] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers deac2568d92f4b5695be7fbd429e7269 (127.8.14.67:36885), ca3113b8417d4a9b867af7236bfd57ea (127.8.14.65:36623)
I20260808 16:58:44.815392 25917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.811482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:36885 (local address 127.8.14.66:36209)
0808 16:58:44.811536 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 16:58:44.811551 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 16:58:44.811614 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:58:44.811863 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:58:44.811864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:58:44.811866 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:58:44.811942 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:58:44.811944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.813081 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.813082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:58:44.814361 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:58:44.814364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.815159 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.815160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:58:44.815230 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.815233 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:58:44.815280 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 16:58:44.815317 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":76}
I20260808 16:58:44.815881 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.811626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:36209 (local address 127.8.14.67:36885)
0808 16:58:44.811680 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 16:58:44.811682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:58:44.811691 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:58:44.811722 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:58:44.811723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:58:44.811734 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:58:44.811803 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:58:44.812097 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.812929 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.814473 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.815054 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.815118 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.815130 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 16:58:44.815770 (+ 640us) server_negotiation.cc:299] Negotiation successful
0808 16:58:44.815821 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 16:58:44.815877 26158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.811573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:42629 (local address 127.8.14.65:36623)
0808 16:58:44.811618 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:58:44.811621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:58:44.811990 (+ 369us) server_negotiation.cc:407] Connection header received
0808 16:58:44.812093 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:58:44.812094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:58:44.812107 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:58:44.812152 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:58:44.812369 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.813064 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.814857 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.815478 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.815530 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.815541 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 16:58:44.815770 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 16:58:44.815813 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:58:44.816108 26174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.811482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:36623 (local address 127.8.14.66:42629)
0808 16:58:44.811930 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 16:58:44.811940 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:58:44.812017 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:58:44.812216 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:58:44.812216 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:58:44.812218 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:58:44.812281 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:58:44.812283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.813171 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.813172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:58:44.814764 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:58:44.814766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.815615 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.815616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:58:44.815682 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.815687 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:58:44.815992 (+ 305us) client_negotiation.cc:240] Negotiation successful
0808 16:58:44.816037 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":150,"thread_start_us":120,"threads_started":1}
I20260808 16:58:44.816149 26038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dd517abcdb5498b8da058f07e65cbae" candidate_uuid: "12eb854662c24e21be8e0436e177cbf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "deac2568d92f4b5695be7fbd429e7269" is_pre_election: true
I20260808 16:58:44.816315 26038 raft_consensus.cc:2454] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12eb854662c24e21be8e0436e177cbf8 in term 0.
I20260808 16:58:44.816430 25649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dd517abcdb5498b8da058f07e65cbae" candidate_uuid: "12eb854662c24e21be8e0436e177cbf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca3113b8417d4a9b867af7236bfd57ea" is_pre_election: true
I20260808 16:58:44.816597 25649 raft_consensus.cc:2454] T 0dd517abcdb5498b8da058f07e65cbae P ca3113b8417d4a9b867af7236bfd57ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12eb854662c24e21be8e0436e177cbf8 in term 0.
I20260808 16:58:44.816584 25757 leader_election.cc:302] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12eb854662c24e21be8e0436e177cbf8, deac2568d92f4b5695be7fbd429e7269; no voters:
I20260808 16:58:44.816699 26173 raft_consensus.cc:2781] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:58:44.816740 26173 raft_consensus.cc:482] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:58:44.816763 26173 raft_consensus.cc:3037] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:58:44.817636 26173 raft_consensus.cc:504] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deac2568d92f4b5695be7fbd429e7269" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 36885 } } peers { permanent_uuid: "12eb854662c24e21be8e0436e177cbf8" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 43023 } } peers { permanent_uuid: "ca3113b8417d4a9b867af7236bfd57ea" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 36623 } }
I20260808 16:58:44.817770 26173 leader_election.cc:288] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [CANDIDATE]: Term 1 election: Requested vote from peers deac2568d92f4b5695be7fbd429e7269 (127.8.14.67:36885), ca3113b8417d4a9b867af7236bfd57ea (127.8.14.65:36623)
I20260808 16:58:44.817970 25649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dd517abcdb5498b8da058f07e65cbae" candidate_uuid: "12eb854662c24e21be8e0436e177cbf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca3113b8417d4a9b867af7236bfd57ea"
I20260808 16:58:44.818063 25649 raft_consensus.cc:3037] T 0dd517abcdb5498b8da058f07e65cbae P ca3113b8417d4a9b867af7236bfd57ea [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:58:44.818073 26038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dd517abcdb5498b8da058f07e65cbae" candidate_uuid: "12eb854662c24e21be8e0436e177cbf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "deac2568d92f4b5695be7fbd429e7269"
I20260808 16:58:44.818148 26038 raft_consensus.cc:3037] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:58:44.818966 25649 raft_consensus.cc:2454] T 0dd517abcdb5498b8da058f07e65cbae P ca3113b8417d4a9b867af7236bfd57ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12eb854662c24e21be8e0436e177cbf8 in term 1.
I20260808 16:58:44.819092 26038 raft_consensus.cc:2454] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12eb854662c24e21be8e0436e177cbf8 in term 1.
I20260808 16:58:44.819170 25756 leader_election.cc:302] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12eb854662c24e21be8e0436e177cbf8, ca3113b8417d4a9b867af7236bfd57ea; no voters:
I20260808 16:58:44.819311 26173 raft_consensus.cc:2781] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:58:44.819490 26173 raft_consensus.cc:686] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [term 1 LEADER]: Becoming Leader. State: Replica: 12eb854662c24e21be8e0436e177cbf8, State: Running, Role: LEADER
I20260808 16:58:44.819593 26173 consensus_queue.cc:237] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deac2568d92f4b5695be7fbd429e7269" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 36885 } } peers { permanent_uuid: "12eb854662c24e21be8e0436e177cbf8" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 43023 } } peers { permanent_uuid: "ca3113b8417d4a9b867af7236bfd57ea" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 36623 } }
I20260808 16:58:44.820478 25393 catalog_manager.cc:5495] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12eb854662c24e21be8e0436e177cbf8 (127.8.14.66). New cstate: current_term: 1 leader_uuid: "12eb854662c24e21be8e0436e177cbf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "deac2568d92f4b5695be7fbd429e7269" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 36885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12eb854662c24e21be8e0436e177cbf8" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 43023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca3113b8417d4a9b867af7236bfd57ea" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 36623 } health_report { overall_health: UNKNOWN } } }
I20260808 16:58:44.829303 26161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:58:44.825821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41056 (local address 127.8.14.66:43023)
0808 16:58:44.825885 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 16:58:44.825887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:58:44.825896 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:58:44.825926 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:58:44.825927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:58:44.825945 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:58:44.826020 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:58:44.826226 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.826984 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.828367 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:58:44.828662 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:58:44.828735 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:58:44.828926 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:58:44.829033 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:58:44.829166 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 16:58:44.829200 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 16:58:44.829217 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 16:58:44.832036 25649 raft_consensus.cc:1261] T 0dd517abcdb5498b8da058f07e65cbae P ca3113b8417d4a9b867af7236bfd57ea [term 1 FOLLOWER]: Refusing update from remote peer 12eb854662c24e21be8e0436e177cbf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:58:44.832396 26173 consensus_queue.cc:1035] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca3113b8417d4a9b867af7236bfd57ea" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 36623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:58:44.834559 26038 raft_consensus.cc:1261] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269 [term 1 FOLLOWER]: Refusing update from remote peer 12eb854662c24e21be8e0436e177cbf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:58:44.834848 26173 consensus_queue.cc:1035] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "deac2568d92f4b5695be7fbd429e7269" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 36885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:58:44.841784 26181 mvcc.cc:205] Tried to move back new op lower bound from 7316309298509578240 to 7316309298461573120. Current Snapshot: MvccSnapshot[applied={T|T < 7316309298509578240}]
I20260808 16:58:44.845777 25393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:58:44.845875 25393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:58:44.846599 25393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ccb4d0c1c75e4efbafb6d303cc60d22b: Sending DeleteTablet for 3 replicas of tablet 0dd517abcdb5498b8da058f07e65cbae
I20260808 16:58:44.846988 26016 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dd517abcdb5498b8da058f07e65cbae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:58:44 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:34348
I20260808 16:58:44.847067 25817 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dd517abcdb5498b8da058f07e65cbae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:58:44 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:41044
I20260808 16:58:44.847293 25617 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dd517abcdb5498b8da058f07e65cbae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:58:44 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:60696
I20260808 16:58:44.847968 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 25519
Aug 08 16:58:44 dist-test-slave-1nl6 krb5kdc[24671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208324, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:58:44.859094 26197 tablet_replica.cc:330] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8: stopping tablet replica
I20260808 16:58:44.859311 26197 raft_consensus.cc:2229] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:58:44.859524 26197 raft_consensus.cc:2258] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:58:44.859740 26194 tablet_replica.cc:330] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269: stopping tablet replica
I20260808 16:58:44.860286 26197 ts_tablet_manager.cc:1905] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:58:44.860460 26194 raft_consensus.cc:2229] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:58:44.860625 26194 raft_consensus.cc:2258] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:58:44.862381 26197 ts_tablet_manager.cc:1918] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:58:44.867738 26194 ts_tablet_manager.cc:1905] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:58:44.882483 26194 ts_tablet_manager.cc:1918] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:58:44.882603 26194 log.cc:1196] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/0dd517abcdb5498b8da058f07e65cbae
I20260808 16:58:44.882889 26194 ts_tablet_manager.cc:1939] T 0dd517abcdb5498b8da058f07e65cbae P deac2568d92f4b5695be7fbd429e7269: Deleting consensus metadata
I20260808 16:58:44.883412 26197 log.cc:1196] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/0dd517abcdb5498b8da058f07e65cbae
I20260808 16:58:44.883473 25370 catalog_manager.cc:4841] TS deac2568d92f4b5695be7fbd429e7269 (127.8.14.67:36885): tablet 0dd517abcdb5498b8da058f07e65cbae (table test-table [id=2689be47e92b44d58c40fb84904f4af2]) successfully deleted
I20260808 16:58:44.883746 26197 ts_tablet_manager.cc:1939] T 0dd517abcdb5498b8da058f07e65cbae P 12eb854662c24e21be8e0436e177cbf8: Deleting consensus metadata
I20260808 16:58:44.884174 25365 catalog_manager.cc:4841] TS 12eb854662c24e21be8e0436e177cbf8 (127.8.14.66:43023): tablet 0dd517abcdb5498b8da058f07e65cbae (table test-table [id=2689be47e92b44d58c40fb84904f4af2]) successfully deleted
W20260808 16:58:44.892372 25366 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.14.65:36623 (error 108)
W20260808 16:58:44.900689 26154 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:58:44.899180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:36623 (local address 127.0.0.1:60710)
0808 16:58:44.899290 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 16:58:44.899302 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:58:44.899346 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:58:44.900617 (+ 1271us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.65:36623: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 16:58:44.900866 25366 catalog_manager.cc:4584] TS ca3113b8417d4a9b867af7236bfd57ea (127.8.14.65:36623): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0dd517abcdb5498b8da058f07e65cbae: Network error: Client connection negotiation failed: client connection to 127.8.14.65:36623: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 16:58:44.903052 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 25721
I20260808 16:58:44.928223 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 25925
I20260808 16:58:44.944626 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 25339
I20260808 16:58:44.958595 8249 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 16:58:45.303503 8249 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 16:58:45.303612 8249 mini_ranger.cc:66] Stopping Ranger...
I20260808 16:58:45.648038 8249 mini_ranger.cc:68] Stopped Ranger
2026-08-08 16:58:45.648 UTC [24719] LOG: received smart shutdown request
2026-08-08 16:58:45.652 UTC [24719] LOG: background worker "logical replication launcher" (PID 24733) exited with exit code 1
2026-08-08 16:58:45.652 UTC [24723] LOG: shutting down
2026-08-08 16:58:45.787 UTC [24719] LOG: database system is shut down
2026-08-08T16:58:45Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48094 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:58:45 dist-test-slave-1nl6 krb5kdc[26210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:58:45 dist-test-slave-1nl6 krb5kdc[26210](info): set up 2 sockets
Aug 08 16:58:45 dist-test-slave-1nl6 krb5kdc[26210](info): commencing operation
krb5kdc: starting...
W20260808 16:58:47.123905 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:58:47 dist-test-slave-1nl6 krb5kdc[26210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:58:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:58:47Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.14.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.14.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.14.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/rangeradmin_127.8.14.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.14.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/rangeradmin_127.8.14.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.14.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.14.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.14.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/rangerlookup_127.8.14.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.14.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/rangerlookup_127.8.14.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.14.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.14.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.14.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/HTTP_127.8.14.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.14.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/HTTP_127.8.14.84@KRBTEST.COM.keytab.
I20260808 16:58:47.284698 8249 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskiOuv3j/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/postgres -l logfile start
2026-08-08 16:58:49.576 UTC [26267] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 16:58:49.576 UTC [26267] LOG: listening on IPv4 address "127.8.14.84", port 59471
2026-08-08 16:58:49.579 UTC [26267] LOG: listening on Unix socket "/tmp/.s.PGSQL.59471"
2026-08-08 16:58:49.600 UTC [26271] LOG: database system was shut down at 2026-08-08 16:58:48 UTC
2026-08-08 16:58:49.603 UTC [26267] LOG: database system is ready to accept connections
I20260808 16:58:50.779958 8249 mini_postgres.cc:96] Postgres bound to 59471
2026-08-08 16:58:50.787 UTC [26282] FATAL: database "slave" does not exist
127.8.14.84:59471 - accepting connections
I20260808 16:58:50.787982 8249 mini_ranger.cc:161] Starting Ranger...
I20260808 16:58:50.803194 8249 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 16:58:51.221231 8249 mini_ranger.cc:87] Created ranger Postgres database
I20260808 16:58:51.221347 8249 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-admin
2026-08-08 16:58:51,505 [I] DB FLAVOR :POSTGRES
2026-08-08 16:58:51,505 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 16:58:51,505 [I] Checking connection..
2026-08-08 16:58:51,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 16:58:51,743 [I] Checking connection passed.
2026-08-08 16:58:51,744 [I] --------- Verifying version history table ---------
2026-08-08 16:58:51,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 16:58:52,004 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 16:58:52,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 16:58:52,262 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 16:58:52,263 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 16:58:52,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 16:58:52.513 UTC [26375] WARNING: there is no transaction in progress
2026-08-08 16:58:52,521 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 16:58:52,522 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 16:58:52,766 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 16:58:52,767 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 16:58:52,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 16:58:53,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 16:58:53,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1nl6.c.gcp-upstream.internal','N') ;"
2026-08-08 16:58:53,502 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 16:58:53,765 [I] Table x_portal_user does not exist in database ranger
2026-08-08 16:58:53,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 16:58:54,017 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 16:58:54,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 16:58:54,268 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 16:58:54,509 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 16:58:54,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 16:58:55.434 UTC [26545] WARNING: there is no transaction in progress
2026-08-08 16:58:55.449 UTC [26545] WARNING: there is no transaction in progress
2026-08-08 16:58:55.464 UTC [26545] WARNING: there is no transaction in progress
2026-08-08 16:58:55.478 UTC [26545] WARNING: there is no transaction in progress
2026-08-08 16:58:55.493 UTC [26545] WARNING: there is no transaction in progress
2026-08-08 16:58:55.509 UTC [26545] WARNING: there is no transaction in progress
2026-08-08 16:58:55.560 UTC [26545] WARNING: there is no transaction in progress
2026-08-08 16:58:55.567 UTC [26545] WARNING: there is no transaction in progress
2026-08-08 16:58:55.580 UTC [26545] WARNING: there is no transaction in progress
2026-08-08 16:58:55.589 UTC [26545] WARNING: there is no transaction in progress
2026-08-08 16:58:55.599 UTC [26545] WARNING: there is no transaction in progress
2026-08-08 16:58:55.611 UTC [26545] WARNING: there is no transaction in progress
2026-08-08 16:58:55.620 UTC [26545] WARNING: there is no transaction in progress
2026-08-08 16:58:55.626 UTC [26545] WARNING: there is no transaction in progress
2026-08-08 16:58:55.632 UTC [26545] WARNING: there is no transaction in progress
2026-08-08 16:58:55.637 UTC [26545] WARNING: there is no transaction in progress
2026-08-08 16:58:56,209 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 16:58:56,209 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 16:58:56,452 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 16:58:56,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 16:58:56,708 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 16:58:56,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 16:58:56,964 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 16:58:56,965 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 16:58:57,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 16:58:57,471 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1nl6.c.gcp-upstream.internal';"
2026-08-08 16:58:57,717 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 16:58:57,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 16:58:57,979 [I] DB_PATCHES have already been applied
I20260808 16:58:57.985075 8249 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-admin:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskiOuv3j/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 16:58:57.985140 8249 mini_ranger.cc:193] Using host: 127.8.14.84
I20260808 16:58:57.989073 8249 mini_ranger.cc:227] Ranger admin URL: http://127.8.14.84:35363
Aug 08, 2026 4:58:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 4:58:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 4:58:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 4:58:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 4:58:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 4:58:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 4:58:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 16:58:59.394793 8249 mini_ranger.cc:160] Time spent starting Ranger: real 8.607s user 0.001s sys 0.009s
Aug 08, 2026 4:59:18 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 4:59:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 4:59:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 4:59:18 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 4:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 4:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 4:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 4:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 4:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 4:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 4:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 4:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 4:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 4:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 4:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 4:59:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 4:59:18 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 4:59:19 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 16:59:20.027376 8249 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.14.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.14.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.14.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/rangerkms_127.8.14.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.14.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/rangerkms_127.8.14.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.14.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.14.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.14.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/HTTP_127.8.14.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.14.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/HTTP_127.8.14.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 16:59:20 dist-test-slave-1nl6 krb5kdc[26210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208360, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 16:59:20.129968 8249 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 16:59:20.143378 8249 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 16:59:20.906020 8249 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 16:59:20.906106 8249 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-kms
I20260808 16:59:20.906124 8249 mini_ranger_kms.cc:227] Using postgres at 127.8.14.84:59471
2026-08-08 16:59:21,070 [I] DB FLAVOR :POSTGRES
2026-08-08 16:59:21,071 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 16:59:21,071 [I] Checking connection
2026-08-08 16:59:21,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 16:59:21,306 [I] connection success
2026-08-08 16:59:21,306 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 16:59:21,306 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 16:59:21,306 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 16:59:21,555 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 16:59:21,555 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 16:59:21,556 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 16:59:21,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.14.84:59471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 16:59:21,815 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 16:59:22.294418 8249 mini_ranger_kms.cc:310] Created kms service
I20260808 16:59:22.506140 8249 mini_ranger_kms.cc:326] Created kudu user
I20260808 16:59:22.550937 8249 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 16:59:22 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208360, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:59:22.928587 8249 mini_ranger_kms.cc:384] Added ranger policy
I20260808 16:59:22.928705 8249 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-kms:/tmp/dist-test-taskiOuv3j/build/release/bin/postgresql.jar:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskiOuv3j/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 16:59:22.928725 8249 mini_ranger_kms.cc:240] Using host: 127.8.14.84
I20260808 16:59:22.933595 8249 mini_ranger_kms.cc:276] Ranger KMS PID: 26816
I20260808 16:59:22.933681 8249 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.14.84:43279
Aug 08, 2026 4:59:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 4:59:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 4:59:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 4:59:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 4:59:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 4:59:24 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 16:59:25 dist-test-slave-1nl6 krb5kdc[26210](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786208365, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.14.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 16:59:25 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208365, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.14.84@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:59:25.693585 8249 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.564s user 0.008s sys 0.140s
I20260808 16:59:25.693701 8249 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 16:59:25.693749 8249 mini_ranger_kms.cc:401] 127.8.14.84:43279/kms/v1/keys
Aug 08 16:59:25 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208365, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.14.84@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
Aug 08 16:59:28 dist-test-slave-1nl6 krb5kdc[26210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:59:28.163774 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:44465
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:44465
--ranger_config_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.14.84:43279/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:59:28.246359 26898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:28.246569 26898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:28.246592 26898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:28.247758 26898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:59:28.247805 26898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:28.247821 26898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:28.247833 26898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:59:28.247888 26898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:59:28.249523 26898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.14.84:43279/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:44465
--ranger_config_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:44465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:28.249770 26898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:28.250013 26898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:28.254767 26912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:28.254782 26909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:28.254817 26910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:28.255069 26898 server_base.cc:1034] running on GCE node
I20260808 16:59:28.255327 26898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:28.255626 26898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:28.256791 26898 hybrid_clock.cc:648] HybridClock initialized: now 1786208368256784 us; error 36 us; skew 500 ppm
Aug 08 16:59:28 dist-test-slave-1nl6 krb5kdc[26210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208368, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:28.263103 26898 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:59:28.263545 26898 webserver.cc:460] Webserver started at http://127.8.14.126:36761/ using document root <none> and password file <none>
I20260808 16:59:28.263773 26898 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:28.263844 26898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:28.264006 26898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 16:59:28 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208368, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:59:28.311074 26898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7"
format_stamp: "Formatted at 2026-08-08 16:59:28 on dist-test-slave-1nl6"
server_key: "cbc4b75d04403eaa60f15d35561490cd"
server_key_iv: "26e95f70fef9a8bc3f7b90aa9d7e469b"
server_key_version: "kuduclusterkey@0"
I20260808 16:59:28.311558 26898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7"
format_stamp: "Formatted at 2026-08-08 16:59:28 on dist-test-slave-1nl6"
server_key: "cbc4b75d04403eaa60f15d35561490cd"
server_key_iv: "26e95f70fef9a8bc3f7b90aa9d7e469b"
server_key_version: "kuduclusterkey@0"
I20260808 16:59:28.313552 26898 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 16:59:28.325448 26923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:28.325618 26898 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:28.325711 26898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7"
format_stamp: "Formatted at 2026-08-08 16:59:28 on dist-test-slave-1nl6"
server_key: "cbc4b75d04403eaa60f15d35561490cd"
server_key_iv: "26e95f70fef9a8bc3f7b90aa9d7e469b"
server_key_version: "kuduclusterkey@0"
I20260808 16:59:28.325817 26898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:28.336266 26898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:28.338366 26898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:28.338541 26898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:28.343641 26898 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:44465
I20260808 16:59:28.343730 27000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:44465 every 8 connection(s)
I20260808 16:59:28.344105 26898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:59:28.344867 27001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:28.347517 27001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7: Bootstrap starting.
I20260808 16:59:28.348356 27001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:28.348778 27001 log.cc:824] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:28.349922 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 26898
I20260808 16:59:28.350032 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:59:28.350251 8249 external_mini_cluster.cc:1428] Setting key e1ee9d772e6a14804adb771f7c3ebae7
I20260808 16:59:28.350540 27001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7: No bootstrap required, opened a new log
I20260808 16:59:28.353360 27001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44465 } }
I20260808 16:59:28.353523 27001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:28.353546 27001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 299a5c74fecd4d179be5cf5cb4a7c7b7, State: Initialized, Role: FOLLOWER
I20260808 16:59:28.353720 27001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44465 } }
I20260808 16:59:28.353799 27001 raft_consensus.cc:388] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:59:28.353835 27001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:59:28.353878 27001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:28.354529 27001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44465 } }
I20260808 16:59:28.354657 27001 leader_election.cc:302] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 299a5c74fecd4d179be5cf5cb4a7c7b7; no voters:
I20260808 16:59:28.354868 27001 leader_election.cc:288] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:59:28.354949 27010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:59:28.355175 27010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [term 1 LEADER]: Becoming Leader. State: Replica: 299a5c74fecd4d179be5cf5cb4a7c7b7, State: Running, Role: LEADER
I20260808 16:59:28.355258 27001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:59:28.355340 27010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44465 } }
I20260808 16:59:28.355707 27010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 299a5c74fecd4d179be5cf5cb4a7c7b7. Latest consensus state: current_term: 1 leader_uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44465 } } }
I20260808 16:59:28.355830 27011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44465 } } }
I20260808 16:59:28.355877 27010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [sys.catalog]: This master's current role is: LEADER
I20260808 16:59:28.356004 27011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [sys.catalog]: This master's current role is: LEADER
I20260808 16:59:28.356247 27015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:59:28.357175 27015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:59:28.357462 27001 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 16:59:28.363255 27015 catalog_manager.cc:1347] Generated new cluster ID: 18122dca1ba3470ab664033fc57ec016
I20260808 16:59:28.363344 27015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:59:28 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:28.370402 27008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:28.350829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45060 (local address 127.8.14.126:44465)
0808 16:59:28.351028 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:28.351039 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:28.355173 (+ 4134us) server_negotiation.cc:407] Connection header received
0808 16:59:28.355435 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:28.355444 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:28.355506 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:28.355629 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:28.356622 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:28.360159 (+ 3537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:28.361341 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:28.362312 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:28.362431 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:28.365679 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:28.365684 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:28.365685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:28.365699 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:28.367948 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:28.368435 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:28.368439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:28.368439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:28.368569 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:28.368832 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:28.368833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:28.368833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:28.369207 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:28.369379 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:28.369958 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 16:59:28.370112 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260808 16:59:28.380810 27015 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:59:28.381556 27015 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:59:28.387755 27015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7: Generated new TSK 0
I20260808 16:59:28.387969 27015 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 16:59:29.103017 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.14.84:43279/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:44465
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:59:29.211374 27070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:29.211691 27070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:29.211760 27070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:29.214108 27070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:29.214213 27070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:29.214306 27070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:59:29.217280 27070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.14.84:43279/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:44465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:29.217572 27070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:29.217842 27070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:29.223708 27076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:29.223706 27078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:29.224377 27080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:29.226413 27070 server_base.cc:1034] running on GCE node
I20260808 16:59:29.226683 27070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:29.227007 27070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:29.228188 27070 hybrid_clock.cc:648] HybridClock initialized: now 1786208369228162 us; error 39 us; skew 500 ppm
Aug 08 16:59:29 dist-test-slave-1nl6 krb5kdc[26210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:29.235674 27070 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:59:29.236234 27070 webserver.cc:460] Webserver started at http://127.8.14.65:35315/ using document root <none> and password file <none>
I20260808 16:59:29.236526 27070 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:29.236585 27070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:29.236723 27070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 16:59:29 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:59:29.252094 27070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "06105348a5f741a0953683e950c80b6d"
format_stamp: "Formatted at 2026-08-08 16:59:29 on dist-test-slave-1nl6"
server_key: "2f06038d50fba8fc25169ca4ba74383b"
server_key_iv: "098d0355410fcb3571fec1a3fed1a32d"
server_key_version: "kuduclusterkey@0"
I20260808 16:59:29.252759 27070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "06105348a5f741a0953683e950c80b6d"
format_stamp: "Formatted at 2026-08-08 16:59:29 on dist-test-slave-1nl6"
server_key: "2f06038d50fba8fc25169ca4ba74383b"
server_key_iv: "098d0355410fcb3571fec1a3fed1a32d"
server_key_version: "kuduclusterkey@0"
I20260808 16:59:29.257366 27070 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260808 16:59:29.276499 27101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:29.276770 27070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:29.276904 27070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "06105348a5f741a0953683e950c80b6d"
format_stamp: "Formatted at 2026-08-08 16:59:29 on dist-test-slave-1nl6"
server_key: "2f06038d50fba8fc25169ca4ba74383b"
server_key_iv: "098d0355410fcb3571fec1a3fed1a32d"
server_key_version: "kuduclusterkey@0"
I20260808 16:59:29.277021 27070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:29.289525 27070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:29.291718 27070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:29.291888 27070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:29.292179 27070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:29.292580 27070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:59:29.292636 27070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:29.292703 27070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:59:29.292747 27070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:29.299672 27070 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:38591
I20260808 16:59:29.300186 27070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:59:29.302425 27286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:38591 every 8 connection(s)
Aug 08 16:59:29 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:29.313913 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:29.303355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:59319 (local address 127.8.14.126:44465)
0808 16:59:29.304413 (+ 1058us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:29.304415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:29.304424 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:59:29.304453 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:29.304453 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:29.304466 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:29.304529 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:29.305301 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:29.306108 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:29.307619 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:29.307920 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:29.307987 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:29.310656 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:29.310657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:29.310657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:29.310665 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:29.312462 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:29.312890 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:29.312891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:29.312891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:29.312956 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:29.313235 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:29.313235 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:29.313236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:29.313374 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:29.313444 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:29.313733 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 16:59:29.313818 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1025,"thread_start_us":90,"threads_started":1}
I20260808 16:59:29.314106 27291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:29.303324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44465 (local address 127.8.14.65:59319)
0808 16:59:29.303529 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:29.303582 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:29.304202 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:29.304660 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:29.304664 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:29.304917 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:29.305188 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:29.305195 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:29.306216 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:29.306217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:29.307517 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:29.307520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:29.308069 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:29.308071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:29.308220 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:29.308839 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:29.308850 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:29.310546 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:29.312559 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:29.312567 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:29.312569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:29.312793 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:29.313032 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:29.313032 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:29.313033 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:29.313156 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:29.313447 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:29.313455 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:29.313601 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 16:59:29.313712 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 16:59:29.313861 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":103,"threads_started":1}
I20260808 16:59:29.314669 27287 heartbeater.cc:347] Connected to a master server at 127.8.14.126:44465
I20260808 16:59:29.314771 27287 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:29.315011 27287 heartbeater.cc:511] Master 127.8.14.126:44465 requested a full tablet report, sending...
I20260808 16:59:29.315104 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 27070
I20260808 16:59:29.315223 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:59:29.315423 8249 external_mini_cluster.cc:1428] Setting key 052c29a77ad182d60f3cb68e905e1211
I20260808 16:59:29.315734 26951 ts_manager.cc:194] Registered new tserver with Master: 06105348a5f741a0953683e950c80b6d (127.8.14.65:38591)
I20260808 16:59:29.316979 26951 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:59319
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 16:59:29.394789 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.14.84:43279/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:44465
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 16:59:29 dist-test-slave-1nl6 krb5kdc[26210](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786208369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 16:59:29.518664 27304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:29.518908 27304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:29.518967 27304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:29.520156 27304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:29.520233 27304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:29.520278 27304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:59:29.521858 27304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.14.84:43279/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:44465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:29.522112 27304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:29.522414 27304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:59:29.528131 27304 server_base.cc:1034] running on GCE node
W20260808 16:59:29.528319 27317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:29.528650 27314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:29.528904 27313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:29.529182 27304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:29.529462 27304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:29.530624 27304 hybrid_clock.cc:648] HybridClock initialized: now 1786208369530599 us; error 40 us; skew 500 ppm
Aug 08 16:59:29 dist-test-slave-1nl6 krb5kdc[26210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:29.533341 27304 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:59:29.533895 27304 webserver.cc:460] Webserver started at http://127.8.14.66:43937/ using document root <none> and password file <none>
I20260808 16:59:29.534147 27304 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:29.534214 27304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:29.534368 27304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 16:59:29 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:59:29.549404 27304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "083a86abe8694982a3fc99c509a819ed"
format_stamp: "Formatted at 2026-08-08 16:59:29 on dist-test-slave-1nl6"
server_key: "a6d025811634e16f8cb0e65231cfec93"
server_key_iv: "845f02306a4b7e05d06ce108ec544e3a"
server_key_version: "kuduclusterkey@0"
I20260808 16:59:29.549836 27304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "083a86abe8694982a3fc99c509a819ed"
format_stamp: "Formatted at 2026-08-08 16:59:29 on dist-test-slave-1nl6"
server_key: "a6d025811634e16f8cb0e65231cfec93"
server_key_iv: "845f02306a4b7e05d06ce108ec544e3a"
server_key_version: "kuduclusterkey@0"
I20260808 16:59:29.551400 27304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:59:29.571038 27329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:29.571308 27304 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:29.571424 27304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "083a86abe8694982a3fc99c509a819ed"
format_stamp: "Formatted at 2026-08-08 16:59:29 on dist-test-slave-1nl6"
server_key: "a6d025811634e16f8cb0e65231cfec93"
server_key_iv: "845f02306a4b7e05d06ce108ec544e3a"
server_key_version: "kuduclusterkey@0"
I20260808 16:59:29.571545 27304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:29.597486 27304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:29.600164 27304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:29.600399 27304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:29.600723 27304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:29.601142 27304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:59:29.601243 27304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:29.601293 27304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:59:29.601357 27304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:29.608741 27304 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:40013
I20260808 16:59:29.609123 27304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:59:29.610285 27505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:40013 every 8 connection(s)
Aug 08 16:59:29 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:29.618217 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 27304
I20260808 16:59:29.618345 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:59:29.618572 8249 external_mini_cluster.cc:1428] Setting key 8cfa0fab3c1ecb45a69acc781be5c6b9
I20260808 16:59:29.619400 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:29.609913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:35319 (local address 127.8.14.126:44465)
0808 16:59:29.609982 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:29.609985 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:29.610247 (+ 262us) server_negotiation.cc:407] Connection header received
0808 16:59:29.610925 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:29.610927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:29.610940 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:29.610989 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:29.611635 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:29.612171 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:29.613326 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:29.613596 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:29.613678 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:29.615808 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:29.615810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:29.615810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:29.615825 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:29.617300 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:29.617818 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:29.617820 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:29.617820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:29.617887 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:29.618199 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:29.618200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:29.618201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:29.618342 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:29.618389 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:29.618811 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 16:59:29.619302 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 16:59:29.620187 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:29.609920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44465 (local address 127.8.14.66:35319)
0808 16:59:29.610107 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:29.610152 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:29.610772 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:29.611061 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:29.611071 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:29.611264 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:29.611514 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:29.611520 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:29.612267 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:29.612267 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:29.613229 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:29.613232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:29.613696 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:29.613697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:29.613827 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:29.614375 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:29.614383 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:29.615702 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:29.617398 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:29.617405 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:29.617407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:29.617705 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:29.617972 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:29.617972 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:29.617973 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:29.618110 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:29.618443 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:29.618450 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:29.618626 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 16:59:29.619759 (+ 1133us) client_negotiation.cc:240] Negotiation successful
0808 16:59:29.619947 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":112,"threads_started":1}
I20260808 16:59:29.620908 27506 heartbeater.cc:347] Connected to a master server at 127.8.14.126:44465
I20260808 16:59:29.621032 27506 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:29.621249 27506 heartbeater.cc:511] Master 127.8.14.126:44465 requested a full tablet report, sending...
I20260808 16:59:29.621712 26951 ts_manager.cc:194] Registered new tserver with Master: 083a86abe8694982a3fc99c509a819ed (127.8.14.66:40013)
I20260808 16:59:29.622219 26951 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:35319
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 16:59:29.649600 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.14.84:43279/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:44465
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:59:29.751672 27511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:29.751972 27511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:29.752009 27511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:29.753723 27511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:29.753784 27511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:29.753825 27511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:59:29.756055 27511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.14.84:43279/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:44465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:29.756319 27511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:29.756582 27511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:29.762184 27516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:29.762272 27517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:29.762585 27519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:29.762591 27511 server_base.cc:1034] running on GCE node
I20260808 16:59:29.762837 27511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:29.763128 27511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:29.764293 27511 hybrid_clock.cc:648] HybridClock initialized: now 1786208369764282 us; error 34 us; skew 500 ppm
Aug 08 16:59:29 dist-test-slave-1nl6 krb5kdc[26210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:29.767164 27511 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:59:29.767704 27511 webserver.cc:460] Webserver started at http://127.8.14.67:35803/ using document root <none> and password file <none>
I20260808 16:59:29.768002 27511 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:29.768102 27511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:29.768283 27511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 16:59:29 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:59:29.792464 27511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "8ed32028ab844296833bfe9400615f1b"
format_stamp: "Formatted at 2026-08-08 16:59:29 on dist-test-slave-1nl6"
server_key: "8fc7dd6377788e525b697be7d3313c6d"
server_key_iv: "4062322a9d959e90adb626e119a93104"
server_key_version: "kuduclusterkey@0"
I20260808 16:59:29.793068 27511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "8ed32028ab844296833bfe9400615f1b"
format_stamp: "Formatted at 2026-08-08 16:59:29 on dist-test-slave-1nl6"
server_key: "8fc7dd6377788e525b697be7d3313c6d"
server_key_iv: "4062322a9d959e90adb626e119a93104"
server_key_version: "kuduclusterkey@0"
I20260808 16:59:29.796525 27511 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 16:59:29.814000 27531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:29.814262 27511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260808 16:59:29.814356 27511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "8ed32028ab844296833bfe9400615f1b"
format_stamp: "Formatted at 2026-08-08 16:59:29 on dist-test-slave-1nl6"
server_key: "8fc7dd6377788e525b697be7d3313c6d"
server_key_iv: "4062322a9d959e90adb626e119a93104"
server_key_version: "kuduclusterkey@0"
I20260808 16:59:29.814442 27511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:29.825865 27511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:29.828399 27511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:29.828570 27511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:29.828879 27511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:29.829322 27511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:59:29.829377 27511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:29.829409 27511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:59:29.829481 27511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:29.845472 27511 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:37399
I20260808 16:59:29.846012 27511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 16:59:29.847616 27715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:37399 every 8 connection(s)
Aug 08 16:59:29 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208369, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:29.856276 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 27511
I20260808 16:59:29.856389 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:59:29.856635 8249 external_mini_cluster.cc:1428] Setting key a5edf7495d52a478714351cdf91b1647
I20260808 16:59:29.859954 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:29.847478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:32815 (local address 127.8.14.126:44465)
0808 16:59:29.847524 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:29.847526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:29.848014 (+ 488us) server_negotiation.cc:407] Connection header received
0808 16:59:29.848681 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:29.848682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:29.848696 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:29.848774 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:29.849607 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:29.850364 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:29.851928 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:29.852270 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:29.852321 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:29.855077 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:29.855079 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:29.855079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:29.855086 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:29.857871 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:29.858816 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:29.858818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:29.858818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:29.858878 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:29.859190 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:29.859191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:29.859191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:29.859319 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:29.859361 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:29.859785 (+ 424us) server_negotiation.cc:299] Negotiation successful
0808 16:59:29.859867 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 16:59:29.860669 27724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:29.847179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44465 (local address 127.8.14.67:32815)
0808 16:59:29.847861 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:29.847902 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:29.848545 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:29.848911 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:29.848916 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:29.849206 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:29.849483 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:29.849489 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:29.850477 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:29.850478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:29.851807 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:29.851811 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:29.852420 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:29.852421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:29.852570 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:29.853206 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:29.853216 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:29.854966 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:29.858416 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:29.858422 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:29.858424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:29.858723 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:29.858972 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:29.858973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:29.858973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:29.859097 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:29.859408 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:29.859413 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:29.859593 (+ 180us) client_negotiation.cc:769] Sending connection context
0808 16:59:29.860260 (+ 667us) client_negotiation.cc:240] Negotiation successful
0808 16:59:29.860426 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":100,"threads_started":1}
I20260808 16:59:29.861109 27720 heartbeater.cc:347] Connected to a master server at 127.8.14.126:44465
I20260808 16:59:29.861259 27720 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:29.861490 27720 heartbeater.cc:511] Master 127.8.14.126:44465 requested a full tablet report, sending...
I20260808 16:59:29.861991 26951 ts_manager.cc:194] Registered new tserver with Master: 8ed32028ab844296833bfe9400615f1b (127.8.14.67:37399)
I20260808 16:59:29.862483 26951 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:32815
I20260808 16:59:29.869189 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:59:29.869985 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 27070
I20260808 16:59:29.887957 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 27304
I20260808 16:59:29.897971 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 27511
I20260808 16:59:29.912245 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 26898
I20260808 16:59:29.923969 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:44465
--webserver_interface=127.8.14.126
--webserver_port=36761
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:44465
--ranger_config_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.14.84:43279/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.14.84:43279/kms with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:59:30.006655 27726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:30.006868 27726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:30.006891 27726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:30.008039 27726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:59:30.008085 27726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:30.008098 27726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:30.008142 27726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:59:30.008170 27726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:59:30.009810 27726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.14.84:43279/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:44465
--ranger_config_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:44465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=36761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:30.010070 27726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:30.010306 27726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:30.015309 27735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:30.015347 27740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:30.015317 27734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:30.015504 27726 server_base.cc:1034] running on GCE node
I20260808 16:59:30.015810 27726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:30.016135 27726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:30.017315 27726 hybrid_clock.cc:648] HybridClock initialized: now 1786208370017306 us; error 38 us; skew 500 ppm
Aug 08 16:59:30 dist-test-slave-1nl6 krb5kdc[26210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:30.019991 27726 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:59:30.020481 27726 webserver.cc:460] Webserver started at http://127.8.14.126:36761/ using document root <none> and password file <none>
I20260808 16:59:30.020668 27726 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:30.020718 27726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 16:59:30 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:59:30.095299 27726 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:30.096088 27750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:30.096299 27726 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:30.096403 27726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7"
format_stamp: "Formatted at 2026-08-08 16:59:28 on dist-test-slave-1nl6"
server_key: "cbc4b75d04403eaa60f15d35561490cd"
server_key_iv: "26e95f70fef9a8bc3f7b90aa9d7e469b"
server_key_version: "kuduclusterkey@0"
I20260808 16:59:30.096778 27726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:30.108393 27726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:30.109975 27726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:30.110142 27726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:30.114511 27726 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:44465
I20260808 16:59:30.114564 27837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:44465 every 8 connection(s)
I20260808 16:59:30.114987 27726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:59:30.116185 27838 sys_catalog.cc:263] Verifying existing consensus state
I20260808 16:59:30.116886 27838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7: Bootstrap starting.
I20260808 16:59:30.120455 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 27726
I20260808 16:59:30.120812 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.14.84:43279/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:38591
--local_ip_for_outbound_sockets=127.8.14.65
--tserver_master_addrs=127.8.14.126:44465
--webserver_port=35315
--webserver_interface=127.8.14.65
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.14.84:43279/kms with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:59:30.125643 27838 log.cc:824] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:30.127558 27838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:59:30.127864 27838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7: Bootstrap complete.
I20260808 16:59:30.130252 27838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44465 } }
I20260808 16:59:30.130501 27838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 299a5c74fecd4d179be5cf5cb4a7c7b7, State: Initialized, Role: FOLLOWER
I20260808 16:59:30.130720 27838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44465 } }
I20260808 16:59:30.130790 27838 raft_consensus.cc:388] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:59:30.130822 27838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:59:30.130918 27838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [term 1 FOLLOWER]: Advancing to term 2
I20260808 16:59:30.131958 27838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44465 } }
I20260808 16:59:30.132050 27838 leader_election.cc:302] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 299a5c74fecd4d179be5cf5cb4a7c7b7; no voters:
I20260808 16:59:30.132306 27838 leader_election.cc:288] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 16:59:30.132364 27846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 16:59:30.132544 27846 raft_consensus.cc:686] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [term 2 LEADER]: Becoming Leader. State: Replica: 299a5c74fecd4d179be5cf5cb4a7c7b7, State: Running, Role: LEADER
I20260808 16:59:30.132575 27838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:59:30.132680 27846 consensus_queue.cc:237] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44465 } }
I20260808 16:59:30.133025 27847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44465 } } }
I20260808 16:59:30.133095 27847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [sys.catalog]: This master's current role is: LEADER
I20260808 16:59:30.133369 27848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 299a5c74fecd4d179be5cf5cb4a7c7b7. Latest consensus state: current_term: 2 leader_uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "299a5c74fecd4d179be5cf5cb4a7c7b7" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44465 } } }
I20260808 16:59:30.133422 27848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7 [sys.catalog]: This master's current role is: LEADER
I20260808 16:59:30.133492 27859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:59:30.134099 27859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:59:30.134327 27859 catalog_manager.cc:1259] Loaded cluster ID: 18122dca1ba3470ab664033fc57ec016
I20260808 16:59:30.134416 27859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:59:30.134511 27838 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 16:59:30.138209 27859 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:59:30.138825 27859 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7: Loaded TSK: 0
I20260808 16:59:30.139164 27859 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 16:59:30.236928 27841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:30.237305 27841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:30.237412 27841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:30.239130 27841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:30.239192 27841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:30.239234 27841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:59:30.241381 27841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.14.84:43279/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:38591
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=35315
--tserver_master_addrs=127.8.14.126:44465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:30.241746 27841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:30.242043 27841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:30.248944 27920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:30.249265 27911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:30.249356 27841 server_base.cc:1034] running on GCE node
W20260808 16:59:30.249622 27908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:30.249929 27841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:30.250197 27841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:30.251374 27841 hybrid_clock.cc:648] HybridClock initialized: now 1786208370251335 us; error 51 us; skew 500 ppm
Aug 08 16:59:30 dist-test-slave-1nl6 krb5kdc[26210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:30.254703 27841 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:59:30.255283 27841 webserver.cc:460] Webserver started at http://127.8.14.65:35315/ using document root <none> and password file <none>
I20260808 16:59:30.255566 27841 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:30.255658 27841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 16:59:30 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:59:30.271145 27841 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 16:59:30.272432 27932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:30.272640 27841 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 16:59:30.272730 27841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "06105348a5f741a0953683e950c80b6d"
format_stamp: "Formatted at 2026-08-08 16:59:29 on dist-test-slave-1nl6"
server_key: "2f06038d50fba8fc25169ca4ba74383b"
server_key_iv: "098d0355410fcb3571fec1a3fed1a32d"
server_key_version: "kuduclusterkey@0"
I20260808 16:59:30.273208 27841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:30.293416 27841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:30.295904 27841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:30.296069 27841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:30.296311 27841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:30.296682 27841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:59:30.296733 27841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:30.296766 27841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:59:30.296831 27841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:30.304188 27841 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:38591
I20260808 16:59:30.304895 27841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:59:30.305415 28109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:38591 every 8 connection(s)
I20260808 16:59:30.313098 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 27841
I20260808 16:59:30.313426 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.14.84:43279/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:40013
--local_ip_for_outbound_sockets=127.8.14.66
--tserver_master_addrs=127.8.14.126:44465
--webserver_port=43937
--webserver_interface=127.8.14.66
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.14.84:43279/kms with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 16:59:30 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:30.326052 28112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:30.306977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44465 (local address 127.8.14.65:49155)
0808 16:59:30.307382 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:30.307426 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:30.308036 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:30.311561 (+ 3525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:30.311566 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:30.311797 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:30.312070 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:30.312076 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:30.313805 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:30.313806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:30.315032 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:30.315035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:30.315870 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:30.315871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:30.316002 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:30.316630 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:30.316639 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:30.318401 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:30.323952 (+ 5551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:30.323961 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:30.323964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:30.324292 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:30.324621 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:30.324622 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:30.324623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:30.324780 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:30.325305 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:30.325312 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:30.325484 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 16:59:30.325580 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 16:59:30.325786 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":87,"threads_started":1}
I20260808 16:59:30.327044 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:30.307073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:49155 (local address 127.8.14.126:44465)
0808 16:59:30.311070 (+ 3997us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:30.311079 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:30.311128 (+ 49us) server_negotiation.cc:407] Connection header received
0808 16:59:30.311281 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:30.311296 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:30.311350 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:30.312228 (+ 878us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:30.312323 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:30.313680 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:30.315144 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:30.315745 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:30.316286 (+ 541us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:30.318540 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:30.318545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:30.318545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:30.318557 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:30.323822 (+ 5265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:30.324411 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:30.324415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:30.324415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:30.324530 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:30.324875 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:30.324875 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:30.324876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:30.325211 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:30.326573 (+ 1362us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:30.326653 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 16:59:30.326795 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3936,"thread_start_us":90,"threads_started":1}
I20260808 16:59:30.328006 28110 heartbeater.cc:347] Connected to a master server at 127.8.14.126:44465
I20260808 16:59:30.328118 28110 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:30.328320 28110 heartbeater.cc:511] Master 127.8.14.126:44465 requested a full tablet report, sending...
W20260808 16:59:30.329452 28110 heartbeater.cc:644] Failed to heartbeat to 127.8.14.126:44465 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 16:59:30.329520 28110 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:30.329581 28110 heartbeater.cc:511] Master 127.8.14.126:44465 requested a full tablet report, sending...
I20260808 16:59:30.329996 28110 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:30.330065 28110 heartbeater.cc:511] Master 127.8.14.126:44465 requested a full tablet report, sending...
W20260808 16:59:30.330387 28110 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 16:59:30.426982 28115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:30.427225 28115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:30.427268 28115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:30.428882 28115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:30.429010 28115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:30.429111 28115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:59:30.431174 28115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.14.84:43279/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:40013
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=43937
--tserver_master_addrs=127.8.14.126:44465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:30.431658 28115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:30.431982 28115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:30.438007 28128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:30.439067 28125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:30.439817 28115 server_base.cc:1034] running on GCE node
W20260808 16:59:30.443081 28126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:30.443361 28115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:30.443655 28115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:30.444839 28115 hybrid_clock.cc:648] HybridClock initialized: now 1786208370444809 us; error 48 us; skew 500 ppm
Aug 08 16:59:30 dist-test-slave-1nl6 krb5kdc[26210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:30.448083 28115 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:59:30.448661 28115 webserver.cc:460] Webserver started at http://127.8.14.66:43937/ using document root <none> and password file <none>
I20260808 16:59:30.448972 28115 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:30.449055 28115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 16:59:30 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:59:30.466820 28115 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260808 16:59:30.468047 28136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:30.468235 28115 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:30.468334 28115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "083a86abe8694982a3fc99c509a819ed"
format_stamp: "Formatted at 2026-08-08 16:59:29 on dist-test-slave-1nl6"
server_key: "a6d025811634e16f8cb0e65231cfec93"
server_key_iv: "845f02306a4b7e05d06ce108ec544e3a"
server_key_version: "kuduclusterkey@0"
I20260808 16:59:30.468834 28115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:30.483575 28115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:30.485751 28115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:30.485932 28115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:30.486217 28115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:30.486631 28115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:59:30.486701 28115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:30.486740 28115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:59:30.486768 28115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:30.494547 28115 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:40013
I20260808 16:59:30.494624 28311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:40013 every 8 connection(s)
I20260808 16:59:30.495090 28115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:59:30.500523 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 28115
I20260808 16:59:30.500857 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.14.84:43279/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:37399
--local_ip_for_outbound_sockets=127.8.14.67
--tserver_master_addrs=127.8.14.126:44465
--webserver_port=35803
--webserver_interface=127.8.14.67
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:41413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.14.84:43279/kms with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 16:59:30 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:30.508163 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:30.495877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44465 (local address 127.8.14.66:57843)
0808 16:59:30.496098 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:30.496152 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:30.496850 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:30.497170 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:30.497183 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:30.497436 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:30.497734 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:30.497740 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:30.498790 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:30.498791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:30.500248 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:30.500251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:30.500935 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:30.500937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:30.501088 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:30.501690 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:30.501700 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:30.503363 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:30.505420 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:30.505428 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:30.505430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:30.505755 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:30.506824 (+ 1069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:30.506825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:30.506826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:30.507003 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:30.507388 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:30.507395 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:30.507577 (+ 182us) client_negotiation.cc:769] Sending connection context
0808 16:59:30.507688 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 16:59:30.507868 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":106,"threads_started":1}
I20260808 16:59:30.508800 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:30.495897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:57843 (local address 127.8.14.126:44465)
0808 16:59:30.495978 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:30.495980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:30.496274 (+ 294us) server_negotiation.cc:407] Connection header received
0808 16:59:30.497008 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:30.497009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:30.497025 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:30.497083 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:30.497859 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:30.498661 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:30.500371 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:30.500720 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:30.500813 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:30.503470 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:30.503471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:30.503472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:30.503481 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:30.505323 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:30.505845 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:30.505846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:30.505846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:30.505907 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:30.507114 (+ 1207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:30.507116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:30.507116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:30.507272 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:30.507355 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:30.508565 (+ 1210us) server_negotiation.cc:299] Negotiation successful
0808 16:59:30.508666 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 16:59:30.509682 28312 heartbeater.cc:347] Connected to a master server at 127.8.14.126:44465
I20260808 16:59:30.509801 28312 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:30.510012 28312 heartbeater.cc:511] Master 127.8.14.126:44465 requested a full tablet report, sending...
W20260808 16:59:30.510874 28312 heartbeater.cc:644] Failed to heartbeat to 127.8.14.126:44465 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 16:59:30.510989 28312 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:30.511063 28312 heartbeater.cc:511] Master 127.8.14.126:44465 requested a full tablet report, sending...
I20260808 16:59:30.511485 28312 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:30.511574 28312 heartbeater.cc:511] Master 127.8.14.126:44465 requested a full tablet report, sending...
W20260808 16:59:30.511909 28312 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 16:59:30.609114 28318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:30.609393 28318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:30.609429 28318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:30.611114 28318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:30.611169 28318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:30.611207 28318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:59:30.613718 28318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.14.84:43279/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:37399
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=35803
--tserver_master_addrs=127.8.14.126:44465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:30.613982 28318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:30.614223 28318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:30.620406 28331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:30.620961 28318 server_base.cc:1034] running on GCE node
W20260808 16:59:30.621376 28327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:30.621552 28326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:30.621842 28318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:30.622112 28318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:30.623279 28318 hybrid_clock.cc:648] HybridClock initialized: now 1786208370623258 us; error 37 us; skew 500 ppm
Aug 08 16:59:30 dist-test-slave-1nl6 krb5kdc[26210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:30.626600 28318 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:59:30.627132 28318 webserver.cc:460] Webserver started at http://127.8.14.67:35803/ using document root <none> and password file <none>
I20260808 16:59:30.627420 28318 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:30.627507 28318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 16:59:30 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:59:30.646036 28318 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:30.647245 28349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:30.647549 28318 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:30.647670 28318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "8ed32028ab844296833bfe9400615f1b"
format_stamp: "Formatted at 2026-08-08 16:59:29 on dist-test-slave-1nl6"
server_key: "8fc7dd6377788e525b697be7d3313c6d"
server_key_iv: "4062322a9d959e90adb626e119a93104"
server_key_version: "kuduclusterkey@0"
I20260808 16:59:30.648165 28318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:30.660558 28318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:30.662613 28318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:30.662775 28318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:30.663058 28318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:30.663651 28318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:59:30.663710 28318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:30.663743 28318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:59:30.663762 28318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:30.671343 28318 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:37399
I20260808 16:59:30.671422 28517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:37399 every 8 connection(s)
I20260808 16:59:30.671773 28318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
Aug 08 16:59:30 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208370, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:30.679154 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 28318
I20260808 16:59:30.682966 28522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:30.672463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44465 (local address 127.8.14.67:37013)
0808 16:59:30.672646 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:30.672690 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:30.673292 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:30.673573 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:30.673577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:30.673792 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:30.674051 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:30.674056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:30.675022 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:30.675023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:30.676235 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:30.676237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:30.676601 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:30.676602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:30.676753 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:30.677281 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:30.677290 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:30.679016 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:30.681215 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:30.681222 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:30.681224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:30.681575 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:30.681885 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:30.681886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:30.681887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:30.682017 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:30.682275 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:30.682281 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:30.682444 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 16:59:30.682558 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 16:59:30.682712 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":90,"threads_started":1}
I20260808 16:59:30.683462 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:30.672498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:37013 (local address 127.8.14.126:44465)
0808 16:59:30.672562 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:30.672565 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:30.672796 (+ 231us) server_negotiation.cc:407] Connection header received
0808 16:59:30.673391 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:30.673392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:30.673403 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:30.673500 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:30.674155 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:30.674888 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:30.676317 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:30.676501 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:30.676553 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:30.679111 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:30.679113 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:30.679113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:30.679120 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:30.681115 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:30.681690 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:30.681692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:30.681692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:30.681759 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:30.682086 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:30.682087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:30.682087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:30.682223 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:30.682283 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:30.683267 (+ 984us) server_negotiation.cc:299] Negotiation successful
0808 16:59:30.683363 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 16:59:30.684170 28518 heartbeater.cc:347] Connected to a master server at 127.8.14.126:44465
I20260808 16:59:30.684280 28518 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:30.684492 28518 heartbeater.cc:511] Master 127.8.14.126:44465 requested a full tablet report, sending...
W20260808 16:59:30.685091 28518 heartbeater.cc:644] Failed to heartbeat to 127.8.14.126:44465 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 16:59:30.685168 28518 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:30.685232 28518 heartbeater.cc:511] Master 127.8.14.126:44465 requested a full tablet report, sending...
I20260808 16:59:30.685564 28518 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:30.685647 28518 heartbeater.cc:511] Master 127.8.14.126:44465 requested a full tablet report, sending...
W20260808 16:59:30.685945 28518 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 16:59:30.690927 28527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:30.679604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45066 (local address 127.8.14.126:44465)
0808 16:59:30.680445 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:30.680447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:30.680454 (+ 7us) server_negotiation.cc:407] Connection header received
0808 16:59:30.680475 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:30.680476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:30.680489 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:30.680559 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:30.681356 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:30.682120 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:30.683742 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:30.684031 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:30.684102 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:30.687781 (+ 3679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:30.687783 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:30.687783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:30.687791 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:30.689564 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:30.689999 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:30.690001 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:30.690001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:30.690066 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:30.690313 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:30.690314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:30.690314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:30.690446 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:30.690619 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:30.690724 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 16:59:30.690808 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":820,"thread_start_us":88,"threads_started":1}
Aug 08 16:59:31 dist-test-slave-1nl6 krb5kdc[26210](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786208371, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:31.330575 28110 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:31.330739 28110 heartbeater.cc:511] Master 127.8.14.126:44465 requested a full tablet report, sending...
I20260808 16:59:31.331228 27779 ts_manager.cc:194] Registered new tserver with Master: 06105348a5f741a0953683e950c80b6d (127.8.14.65:38591)
I20260808 16:59:31.332123 27779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:49155
I20260808 16:59:31.512071 28312 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:31.512254 28312 heartbeater.cc:511] Master 127.8.14.126:44465 requested a full tablet report, sending...
I20260808 16:59:31.512543 27779 ts_manager.cc:194] Registered new tserver with Master: 083a86abe8694982a3fc99c509a819ed (127.8.14.66:40013)
I20260808 16:59:31.513096 27779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:57843
Aug 08 16:59:31 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208371, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:59:31.686113 28518 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:31.686267 28518 heartbeater.cc:511] Master 127.8.14.126:44465 requested a full tablet report, sending...
I20260808 16:59:31.686630 27779 ts_manager.cc:194] Registered new tserver with Master: 8ed32028ab844296833bfe9400615f1b (127.8.14.67:37399)
I20260808 16:59:31.687233 27779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:37013
I20260808 16:59:31.688679 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:59:31.698143 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:31.689692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45080 (local address 127.8.14.126:44465)
0808 16:59:31.690021 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:31.690023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:31.690131 (+ 108us) server_negotiation.cc:407] Connection header received
0808 16:59:31.690229 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:31.690229 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:31.690241 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:31.690285 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:31.691056 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.691824 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.692917 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.693233 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.693274 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:31.694845 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:31.694846 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:31.694847 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:31.694854 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:31.696859 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:31.697260 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:31.697261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:31.697262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:31.697320 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:31.697520 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:31.697521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:31.697521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:31.697647 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:31.697719 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:31.697931 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 16:59:31.698034 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":78,"threads_started":1}
I20260808 16:59:31.700958 27779 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:59:31.703756 27779 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:59:31.723381 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:31.712650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:40013 (local address 127.0.0.1:47184)
0808 16:59:31.714050 (+ 1400us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:31.714094 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:31.714170 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:31.716584 (+ 2414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:31.716585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:31.716588 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:31.716641 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:31.716643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.720734 (+ 4091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.720734 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:31.721803 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:31.721806 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.723040 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.723041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:31.723121 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:31.723135 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 16:59:31.723214 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 16:59:31.723272 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1353,"thread_start_us":77,"threads_started":1}
I20260808 16:59:31.724190 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:31.712743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47184 (local address 127.8.14.66:40013)
0808 16:59:31.713968 (+ 1225us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:31.713972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:31.714260 (+ 288us) server_negotiation.cc:407] Connection header received
0808 16:59:31.714301 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:31.714303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:31.714321 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:31.714386 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:31.716733 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.718089 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.721904 (+ 3815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.722683 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.722757 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:31.722824 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:31.723745 (+ 921us) server_negotiation.cc:299] Negotiation successful
0808 16:59:31.723796 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1189,"thread_start_us":57,"threads_started":1}
I20260808 16:59:31.724963 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:31.712970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33836 (local address 127.8.14.67:37399)
0808 16:59:31.713858 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:31.713863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:31.714660 (+ 797us) server_negotiation.cc:407] Connection header received
0808 16:59:31.714699 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:31.714702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:31.714717 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:31.714790 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:31.715093 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.716482 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.719653 (+ 3171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.722857 (+ 3204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.722935 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:31.722996 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:31.724839 (+ 1843us) server_negotiation.cc:299] Negotiation successful
0808 16:59:31.724884 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":850,"thread_start_us":40,"threads_started":1}
I20260808 16:59:31.724768 28561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:31.712879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:37399 (local address 127.0.0.1:33836)
0808 16:59:31.714521 (+ 1642us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:31.714532 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:31.714583 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:31.714928 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:31.714931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:31.714934 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:31.715007 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:31.715009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.718181 (+ 3172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.718182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:31.719537 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:31.719540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.724510 (+ 4970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.724512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:31.724578 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:31.724582 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:59:31.724653 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 16:59:31.724699 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1609,"thread_start_us":163,"threads_started":1}
I20260808 16:59:31.726030 28177 tablet_service.cc:1467] Processing CreateTablet for tablet 9916b219d30648b985f2ef3adadf9805 (DEFAULT_TABLE table=test-table [id=cdb01f547946450891fab5e14036a71a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:59:31.726442 28177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9916b219d30648b985f2ef3adadf9805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:31.727478 28423 tablet_service.cc:1467] Processing CreateTablet for tablet 9916b219d30648b985f2ef3adadf9805 (DEFAULT_TABLE table=test-table [id=cdb01f547946450891fab5e14036a71a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:59:31.727815 28423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9916b219d30648b985f2ef3adadf9805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:31.727017 28562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:31.713184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:38591 (local address 127.0.0.1:59750)
0808 16:59:31.722979 (+ 9795us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:31.722991 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:31.723056 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:31.723418 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:31.723419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:31.723421 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:31.723461 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:31.723463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.724660 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.724661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:31.725746 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:31.725749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.726743 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.726744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:31.726833 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:31.726837 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:59:31.726891 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 16:59:31.726947 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9775,"thread_start_us":69,"threads_started":1}
I20260808 16:59:31.731886 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:31.713361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59750 (local address 127.8.14.65:38591)
0808 16:59:31.723170 (+ 9809us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:31.723175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:31.723185 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:59:31.723220 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:31.723222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:31.723238 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:31.723325 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:31.723535 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.724562 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.725860 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.726586 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.726656 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:31.726706 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:31.731747 (+ 5041us) server_negotiation.cc:299] Negotiation successful
0808 16:59:31.731807 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9774,"thread_start_us":70,"threads_started":1}
I20260808 16:59:31.732378 28571 tablet_bootstrap.cc:492] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b: Bootstrap starting.
I20260808 16:59:31.732648 27976 tablet_service.cc:1467] Processing CreateTablet for tablet 9916b219d30648b985f2ef3adadf9805 (DEFAULT_TABLE table=test-table [id=cdb01f547946450891fab5e14036a71a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:59:31.732985 27976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9916b219d30648b985f2ef3adadf9805. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:31.733101 28571 tablet_bootstrap.cc:654] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:31.733681 28571 log.cc:824] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:31.734470 28571 tablet_bootstrap.cc:492] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b: No bootstrap required, opened a new log
I20260808 16:59:31.734534 28571 ts_tablet_manager.cc:1397] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 16:59:31.737172 28571 raft_consensus.cc:348] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06105348a5f741a0953683e950c80b6d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38591 } } peers { permanent_uuid: "083a86abe8694982a3fc99c509a819ed" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40013 } } peers { permanent_uuid: "8ed32028ab844296833bfe9400615f1b" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 37399 } }
I20260808 16:59:31.737442 28571 raft_consensus.cc:374] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:31.737527 28571 raft_consensus.cc:729] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ed32028ab844296833bfe9400615f1b, State: Initialized, Role: FOLLOWER
I20260808 16:59:31.737709 28571 consensus_queue.cc:260] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06105348a5f741a0953683e950c80b6d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38591 } } peers { permanent_uuid: "083a86abe8694982a3fc99c509a819ed" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40013 } } peers { permanent_uuid: "8ed32028ab844296833bfe9400615f1b" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 37399 } }
I20260808 16:59:31.738099 28571 ts_tablet_manager.cc:1428] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 16:59:31.738253 28518 heartbeater.cc:503] Master 127.8.14.126:44465 was elected leader, sending a full tablet report...
I20260808 16:59:31.743098 28575 tablet_bootstrap.cc:492] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d: Bootstrap starting.
I20260808 16:59:31.743326 28569 tablet_bootstrap.cc:492] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed: Bootstrap starting.
I20260808 16:59:31.744055 28569 tablet_bootstrap.cc:654] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:31.744760 28575 tablet_bootstrap.cc:654] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:31.745131 28569 log.cc:824] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:31.745517 28575 log.cc:824] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:31.746817 28569 tablet_bootstrap.cc:492] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed: No bootstrap required, opened a new log
I20260808 16:59:31.746966 28569 ts_tablet_manager.cc:1397] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 16:59:31.749944 28569 raft_consensus.cc:348] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06105348a5f741a0953683e950c80b6d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38591 } } peers { permanent_uuid: "083a86abe8694982a3fc99c509a819ed" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40013 } } peers { permanent_uuid: "8ed32028ab844296833bfe9400615f1b" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 37399 } }
I20260808 16:59:31.750172 28569 raft_consensus.cc:374] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:31.750254 28569 raft_consensus.cc:729] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 083a86abe8694982a3fc99c509a819ed, State: Initialized, Role: FOLLOWER
I20260808 16:59:31.750450 28569 consensus_queue.cc:260] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06105348a5f741a0953683e950c80b6d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38591 } } peers { permanent_uuid: "083a86abe8694982a3fc99c509a819ed" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40013 } } peers { permanent_uuid: "8ed32028ab844296833bfe9400615f1b" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 37399 } }
I20260808 16:59:31.750784 28569 ts_tablet_manager.cc:1428] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 16:59:31.750977 28312 heartbeater.cc:503] Master 127.8.14.126:44465 was elected leader, sending a full tablet report...
I20260808 16:59:31.751659 28575 tablet_bootstrap.cc:492] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d: No bootstrap required, opened a new log
I20260808 16:59:31.751801 28575 ts_tablet_manager.cc:1397] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260808 16:59:31.754379 28575 raft_consensus.cc:348] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06105348a5f741a0953683e950c80b6d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38591 } } peers { permanent_uuid: "083a86abe8694982a3fc99c509a819ed" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40013 } } peers { permanent_uuid: "8ed32028ab844296833bfe9400615f1b" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 37399 } }
I20260808 16:59:31.754595 28575 raft_consensus.cc:374] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:31.754675 28575 raft_consensus.cc:729] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06105348a5f741a0953683e950c80b6d, State: Initialized, Role: FOLLOWER
I20260808 16:59:31.754873 28575 consensus_queue.cc:260] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06105348a5f741a0953683e950c80b6d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38591 } } peers { permanent_uuid: "083a86abe8694982a3fc99c509a819ed" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40013 } } peers { permanent_uuid: "8ed32028ab844296833bfe9400615f1b" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 37399 } }
I20260808 16:59:31.755254 28575 ts_tablet_manager.cc:1428] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:59:31.755419 28110 heartbeater.cc:503] Master 127.8.14.126:44465 was elected leader, sending a full tablet report...
W20260808 16:59:31.806660 28111 tablet.cc:2369] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 16:59:31 dist-test-slave-1nl6 krb5kdc[26210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208371, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for HTTP/127.8.14.84@KRBTEST.COM
I20260808 16:59:31.872424 28582 raft_consensus.cc:482] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:59:31.872575 28582 raft_consensus.cc:504] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06105348a5f741a0953683e950c80b6d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38591 } } peers { permanent_uuid: "083a86abe8694982a3fc99c509a819ed" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40013 } } peers { permanent_uuid: "8ed32028ab844296833bfe9400615f1b" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 37399 } }
I20260808 16:59:31.873005 28582 leader_election.cc:288] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 083a86abe8694982a3fc99c509a819ed (127.8.14.66:40013), 8ed32028ab844296833bfe9400615f1b (127.8.14.67:37399)
I20260808 16:59:31.884308 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:31.873909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:49525 (local address 127.8.14.67:37399)
0808 16:59:31.873974 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:31.873977 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:31.877729 (+ 3752us) server_negotiation.cc:407] Connection header received
0808 16:59:31.880716 (+ 2987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:31.880717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:31.880730 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:31.880758 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:31.880940 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.881596 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.883238 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.883912 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.883961 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:31.883971 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:31.884196 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 16:59:31.884238 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 16:59:31.884797 28584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:31.873556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:37399 (local address 127.8.14.65:49525)
0808 16:59:31.877666 (+ 4110us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:31.877677 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:31.880627 (+ 2950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:31.880814 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:31.880815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:31.880816 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:31.880869 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:31.880870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.881909 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.881909 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:31.883108 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:31.883112 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.884053 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.884054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:31.884114 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:31.884117 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:59:31.884653 (+ 536us) client_negotiation.cc:240] Negotiation successful
0808 16:59:31.884696 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4092,"thread_start_us":68,"threads_started":1}
I20260808 16:59:31.885367 28448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9916b219d30648b985f2ef3adadf9805" candidate_uuid: "06105348a5f741a0953683e950c80b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ed32028ab844296833bfe9400615f1b" is_pre_election: true
I20260808 16:59:31.885533 28448 raft_consensus.cc:2454] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06105348a5f741a0953683e950c80b6d in term 0.
I20260808 16:59:31.886025 27934 leader_election.cc:302] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06105348a5f741a0953683e950c80b6d, 8ed32028ab844296833bfe9400615f1b; no voters:
I20260808 16:59:31.886272 28582 raft_consensus.cc:2781] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:59:31.886346 28582 raft_consensus.cc:482] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:59:31.886386 28582 raft_consensus.cc:3037] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:31.887398 28582 raft_consensus.cc:504] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06105348a5f741a0953683e950c80b6d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38591 } } peers { permanent_uuid: "083a86abe8694982a3fc99c509a819ed" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40013 } } peers { permanent_uuid: "8ed32028ab844296833bfe9400615f1b" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 37399 } }
I20260808 16:59:31.887524 28582 leader_election.cc:288] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [CANDIDATE]: Term 1 election: Requested vote from peers 083a86abe8694982a3fc99c509a819ed (127.8.14.66:40013), 8ed32028ab844296833bfe9400615f1b (127.8.14.67:37399)
I20260808 16:59:31.887845 28448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9916b219d30648b985f2ef3adadf9805" candidate_uuid: "06105348a5f741a0953683e950c80b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ed32028ab844296833bfe9400615f1b"
I20260808 16:59:31.887928 28448 raft_consensus.cc:3037] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:31.888751 28448 raft_consensus.cc:2454] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06105348a5f741a0953683e950c80b6d in term 1.
I20260808 16:59:31.888978 27934 leader_election.cc:302] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06105348a5f741a0953683e950c80b6d, 8ed32028ab844296833bfe9400615f1b; no voters:
I20260808 16:59:31.889091 28582 raft_consensus.cc:2781] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:59:31.889230 28582 raft_consensus.cc:686] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [term 1 LEADER]: Becoming Leader. State: Replica: 06105348a5f741a0953683e950c80b6d, State: Running, Role: LEADER
I20260808 16:59:31.889315 28582 consensus_queue.cc:237] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06105348a5f741a0953683e950c80b6d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38591 } } peers { permanent_uuid: "083a86abe8694982a3fc99c509a819ed" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40013 } } peers { permanent_uuid: "8ed32028ab844296833bfe9400615f1b" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 37399 } }
I20260808 16:59:31.889796 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:31.873987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:46971 (local address 127.8.14.66:40013)
0808 16:59:31.874052 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:31.874054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:31.875308 (+ 1254us) server_negotiation.cc:407] Connection header received
0808 16:59:31.875418 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:31.875419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:31.875432 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:31.875460 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:31.877792 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.878581 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.880337 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.880861 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.880934 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:31.880945 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:31.889673 (+ 8728us) server_negotiation.cc:299] Negotiation successful
0808 16:59:31.889729 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 16:59:31.890034 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:31.873309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:40013 (local address 127.8.14.65:46971)
0808 16:59:31.875241 (+ 1932us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:31.875254 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:31.875339 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:31.876236 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:31.876237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:31.876240 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:31.876327 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:31.876329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.878997 (+ 2668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.878998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:31.880240 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:31.880243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.889234 (+ 8991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.889235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:31.889313 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:31.889337 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 16:59:31.889904 (+ 567us) client_negotiation.cc:240] Negotiation successful
0808 16:59:31.889949 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1908,"thread_start_us":146,"threads_started":1}
I20260808 16:59:31.890184 27779 catalog_manager.cc:5495] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 06105348a5f741a0953683e950c80b6d (127.8.14.65). New cstate: current_term: 1 leader_uuid: "06105348a5f741a0953683e950c80b6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06105348a5f741a0953683e950c80b6d" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "083a86abe8694982a3fc99c509a819ed" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ed32028ab844296833bfe9400615f1b" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 37399 } health_report { overall_health: UNKNOWN } } }
I20260808 16:59:31.891114 28244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9916b219d30648b985f2ef3adadf9805" candidate_uuid: "06105348a5f741a0953683e950c80b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "083a86abe8694982a3fc99c509a819ed"
I20260808 16:59:31.891237 28244 raft_consensus.cc:3037] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:31.891134 28214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9916b219d30648b985f2ef3adadf9805" candidate_uuid: "06105348a5f741a0953683e950c80b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "083a86abe8694982a3fc99c509a819ed" is_pre_election: true
I20260808 16:59:31.895104 28244 raft_consensus.cc:2454] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06105348a5f741a0953683e950c80b6d in term 1.
I20260808 16:59:31.899363 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:31.896045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59764 (local address 127.8.14.65:38591)
0808 16:59:31.896103 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:31.896105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:31.896215 (+ 110us) server_negotiation.cc:407] Connection header received
0808 16:59:31.896250 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:31.896251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:31.896271 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:31.896336 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:59:31.896522 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.897185 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.898260 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:31.898537 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:31.898602 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:31.898801 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:59:31.898944 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:59:31.899030 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:31.899267 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 16:59:31.899285 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 16:59:31.901823 28244 raft_consensus.cc:1261] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed [term 1 FOLLOWER]: Refusing update from remote peer 06105348a5f741a0953683e950c80b6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:59:31.901850 28448 raft_consensus.cc:1261] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b [term 1 FOLLOWER]: Refusing update from remote peer 06105348a5f741a0953683e950c80b6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:59:31.902199 28582 consensus_queue.cc:1035] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "083a86abe8694982a3fc99c509a819ed" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 40013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:59:31.902354 28582 consensus_queue.cc:1035] T 9916b219d30648b985f2ef3adadf9805 P 06105348a5f741a0953683e950c80b6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ed32028ab844296833bfe9400615f1b" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 37399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:59:31.909613 28597 mvcc.cc:205] Tried to move back new op lower bound from 7316309491304669184 to 7316309491259289600. Current Snapshot: MvccSnapshot[applied={T|T < 7316309491304669184}]
I20260808 16:59:31.913843 27778 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:59:31.913947 27778 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:59:31.914814 27778 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 299a5c74fecd4d179be5cf5cb4a7c7b7: Sending DeleteTablet for 3 replicas of tablet 9916b219d30648b985f2ef3adadf9805
I20260808 16:59:31.915201 27976 tablet_service.cc:1514] Processing DeleteTablet for tablet 9916b219d30648b985f2ef3adadf9805 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:59:31 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:59750
I20260808 16:59:31.915421 28177 tablet_service.cc:1514] Processing DeleteTablet for tablet 9916b219d30648b985f2ef3adadf9805 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:59:31 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:47184
I20260808 16:59:31.915433 28423 tablet_service.cc:1514] Processing DeleteTablet for tablet 9916b219d30648b985f2ef3adadf9805 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:59:31 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:33836
I20260808 16:59:31.915650 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 27841
I20260808 16:59:31.923044 28608 tablet_replica.cc:330] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed: stopping tablet replica
I20260808 16:59:31.923238 28608 raft_consensus.cc:2229] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:59:31.923425 28608 raft_consensus.cc:2258] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:59:31.925405 28608 ts_tablet_manager.cc:1905] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:59:31.930300 28611 tablet_replica.cc:330] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b: stopping tablet replica
I20260808 16:59:31.930872 28611 raft_consensus.cc:2229] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:59:31.931282 28608 ts_tablet_manager.cc:1918] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:59:31.931437 28608 log.cc:1196] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/9916b219d30648b985f2ef3adadf9805
I20260808 16:59:31.931833 28608 ts_tablet_manager.cc:1939] T 9916b219d30648b985f2ef3adadf9805 P 083a86abe8694982a3fc99c509a819ed: Deleting consensus metadata
I20260808 16:59:31.932474 27760 catalog_manager.cc:4841] TS 083a86abe8694982a3fc99c509a819ed (127.8.14.66:40013): tablet 9916b219d30648b985f2ef3adadf9805 (table test-table [id=cdb01f547946450891fab5e14036a71a]) successfully deleted
I20260808 16:59:31.934198 28611 raft_consensus.cc:2258] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 16:59:31.934644 27757 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.14.65:38591 (error 108)
I20260808 16:59:31.935489 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 28115
I20260808 16:59:31.936241 28611 ts_tablet_manager.cc:1905] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:59:31.941378 28611 ts_tablet_manager.cc:1918] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:59:31.942490 28611 log.cc:1196] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/9916b219d30648b985f2ef3adadf9805
I20260808 16:59:31.943058 28611 ts_tablet_manager.cc:1939] T 9916b219d30648b985f2ef3adadf9805 P 8ed32028ab844296833bfe9400615f1b: Deleting consensus metadata
I20260808 16:59:31.943632 27760 catalog_manager.cc:4841] TS 8ed32028ab844296833bfe9400615f1b (127.8.14.67:37399): tablet 9916b219d30648b985f2ef3adadf9805 (table test-table [id=cdb01f547946450891fab5e14036a71a]) successfully deleted
I20260808 16:59:31.950546 28562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:31.950409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:38591 (local address 127.0.0.1:59772)
0808 16:59:31.950459 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:31.950503 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.65:38591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 16:59:31.950584 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 28318
W20260808 16:59:31.950699 27757 catalog_manager.cc:4584] TS 06105348a5f741a0953683e950c80b6d (127.8.14.65:38591): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9916b219d30648b985f2ef3adadf9805: Network error: Client connection negotiation failed: client connection to 127.8.14.65:38591: connect: Connection refused (error 111)
I20260808 16:59:31.962733 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 27726
I20260808 16:59:31.977020 8249 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 16:59:32.322593 8249 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 16:59:32.322685 8249 mini_ranger.cc:66] Stopping Ranger...
I20260808 16:59:32.667575 8249 mini_ranger.cc:68] Stopped Ranger
2026-08-08 16:59:32.667 UTC [26267] LOG: received smart shutdown request
2026-08-08 16:59:32.671 UTC [26267] LOG: background worker "logical replication launcher" (PID 26279) exited with exit code 1
2026-08-08 16:59:32.672 UTC [26272] LOG: shutting down
2026-08-08 16:59:32.808 UTC [26267] LOG: database system is shut down
2026-08-08T16:59:32Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47015 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:59:32 dist-test-slave-1nl6 krb5kdc[28625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:59:32 dist-test-slave-1nl6 krb5kdc[28625](info): set up 2 sockets
Aug 08 16:59:32 dist-test-slave-1nl6 krb5kdc[28625](info): commencing operation
krb5kdc: starting...
W20260808 16:59:34.126698 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:59:34 dist-test-slave-1nl6 krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:59:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:59:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:59:34.249465 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:44133
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:44133,127.8.14.125:37387,127.8.14.124:40661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:59:34.330371 28645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:34.330607 28645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:34.330646 28645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:34.331849 28645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:59:34.331918 28645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:34.331941 28645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:34.331962 28645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:59:34.331993 28645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:59:34.333472 28645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:44133,127.8.14.125:37387,127.8.14.124:40661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:44133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:34.333766 28645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:34.334014 28645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:34.339059 28653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:34.339068 28651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:34.339102 28645 server_base.cc:1034] running on GCE node
W20260808 16:59:34.339069 28650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:34.339553 28645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:34.339803 28645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:34.340948 28645 hybrid_clock.cc:648] HybridClock initialized: now 1786208374340936 us; error 30 us; skew 500 ppm
Aug 08 16:59:34 dist-test-slave-1nl6 krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:34.346276 28645 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:59:34.346730 28645 webserver.cc:460] Webserver started at http://127.8.14.126:33167/ using document root <none> and password file <none>
I20260808 16:59:34.346977 28645 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:34.347026 28645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:34.347113 28645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:59:34.348083 28645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "a0bfb5a1b6374fbcbf323da5878159e1"
format_stamp: "Formatted at 2026-08-08 16:59:34 on dist-test-slave-1nl6"
server_key: "7ba78e7eed0489ecadd4ebc9edf01d12"
server_key_iv: "1ac2165407d916fbe746ec7fc2c8dd89"
server_key_version: "encryptionkey@0"
I20260808 16:59:34.348381 28645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "a0bfb5a1b6374fbcbf323da5878159e1"
format_stamp: "Formatted at 2026-08-08 16:59:34 on dist-test-slave-1nl6"
server_key: "7ba78e7eed0489ecadd4ebc9edf01d12"
server_key_iv: "1ac2165407d916fbe746ec7fc2c8dd89"
server_key_version: "encryptionkey@0"
I20260808 16:59:34.349635 28645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 16:59:34.350310 28664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:34.350484 28645 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:59:34.350595 28645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "a0bfb5a1b6374fbcbf323da5878159e1"
format_stamp: "Formatted at 2026-08-08 16:59:34 on dist-test-slave-1nl6"
server_key: "7ba78e7eed0489ecadd4ebc9edf01d12"
server_key_iv: "1ac2165407d916fbe746ec7fc2c8dd89"
server_key_version: "encryptionkey@0"
I20260808 16:59:34.350673 28645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:34.358429 28645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:34.360221 28645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:34.360404 28645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:34.364611 28645 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:44133
I20260808 16:59:34.364645 28741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:44133 every 8 connection(s)
I20260808 16:59:34.364976 28645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:59:34.365824 28742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:34.367452 28742 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 16:59:34 dist-test-slave-1nl6 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:34.374015 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 28645
I20260808 16:59:34.374133 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:59:34.374377 8249 external_mini_cluster.cc:1428] Setting key 518da454c72ea3c687fec1e3c7da3738
I20260808 16:59:34.377246 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.368153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55010 (local address 127.8.14.126:44133)
0808 16:59:34.368323 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:34.368331 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:34.368455 (+ 124us) server_negotiation.cc:407] Connection header received
0808 16:59:34.368919 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:34.368920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:34.368937 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:34.369018 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:34.369531 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.370315 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.371168 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.371520 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.371600 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.373706 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:34.373713 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:34.373713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:34.373719 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:34.375720 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.376099 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.376102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.376102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.376212 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.376500 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.376501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.376501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.376702 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:34.376765 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:34.377001 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 16:59:34.377095 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260808 16:59:34.377375 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.368146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44133 (local address 127.0.0.1:55010)
0808 16:59:34.368322 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.368361 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:34.368800 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:34.369046 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:34.369048 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:34.369298 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:34.369446 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.369450 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.370409 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.370410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.371101 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.371103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.371598 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.371599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.371654 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.372203 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:34.372211 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:34.373628 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:34.375821 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.375828 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.375830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.376019 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.376292 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.376293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.376293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.376420 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.376772 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:34.376778 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:34.376864 (+ 86us) client_negotiation.cc:769] Sending connection context
0808 16:59:34.376969 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 16:59:34.377215 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":71,"thread_start_us":101,"threads_started":1}
I20260808 16:59:34.378144 28742 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:59:34.378508 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.378371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:37387 (local address 127.0.0.1:37958)
0808 16:59:34.378423 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.378476 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:37387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 16:59:34.378639 28665 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.14.125:37387: connect: Connection refused (error 111)
I20260808 16:59:34.379066 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.378990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:37387 (local address 127.0.0.1:37970)
0808 16:59:34.379021 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.379048 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:37387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260808 16:59:34.379204 28742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.125:37387: Network error: Client connection negotiation failed: client connection to 127.8.14.125:37387: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.8.14.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.125.keytab.
Entry for principal kudu/127.8.14.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.125.keytab.
I20260808 16:59:34.399505 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.125:37387
--webserver_interface=127.8.14.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.125.keytab
--principal=kudu/127.8.14.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:44133,127.8.14.125:37387,127.8.14.124:40661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:59:34.428455 28742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } attempt: 1
I20260808 16:59:34.428932 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.428796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:37387 (local address 127.0.0.1:37972)
0808 16:59:34.428868 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.428904 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:37387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 16:59:34.429277 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.429180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:37387 (local address 127.0.0.1:37976)
0808 16:59:34.429227 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.429253 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:37387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 16:59:34.429370 28742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.125:37387: Network error: Client connection negotiation failed: client connection to 127.8.14.125:37387: connect: Connection refused (error 111)
W20260808 16:59:34.480968 28750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:34.481208 28750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:34.481247 28750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:34.482398 28750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:59:34.482466 28750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:34.482492 28750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:34.482524 28750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:59:34.482573 28750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:59:34.484126 28750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:44133,127.8.14.125:37387,127.8.14.124:40661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.125.keytab
--principal=kudu/127.8.14.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.125:37387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.14.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:34.484416 28750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:34.484642 28750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:34.489614 28758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:34.489621 28763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:34.489817 28750 server_base.cc:1034] running on GCE node
W20260808 16:59:34.489612 28760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:34.490096 28750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:34.490391 28750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:34.491552 28750 hybrid_clock.cc:648] HybridClock initialized: now 1786208374491542 us; error 28 us; skew 500 ppm
Aug 08 16:59:34 dist-test-slave-1nl6 krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:34.493952 28750 init.cc:377] Logged in from keytab as kudu/127.8.14.125@KRBTEST.COM (short username kudu)
I20260808 16:59:34.494370 28750 webserver.cc:460] Webserver started at http://127.8.14.125:46501/ using document root <none> and password file <none>
I20260808 16:59:34.494589 28750 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:34.494657 28750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:34.494818 28750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:59:34.495872 28750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-1/data/instance:
uuid: "33e5574881eb43de8fca65edf5c4e74d"
format_stamp: "Formatted at 2026-08-08 16:59:34 on dist-test-slave-1nl6"
server_key: "0276d2cd9ccae16081fd705b5cc04d60"
server_key_iv: "1f4ea0c58eb0a18b9349d4875e032710"
server_key_version: "encryptionkey@0"
I20260808 16:59:34.496243 28750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-1/wal/instance:
uuid: "33e5574881eb43de8fca65edf5c4e74d"
format_stamp: "Formatted at 2026-08-08 16:59:34 on dist-test-slave-1nl6"
server_key: "0276d2cd9ccae16081fd705b5cc04d60"
server_key_iv: "1f4ea0c58eb0a18b9349d4875e032710"
server_key_version: "encryptionkey@0"
I20260808 16:59:34.497507 28742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } attempt: 2
I20260808 16:59:34.497560 28750 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:59:34.497874 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.497750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:37387 (local address 127.0.0.1:37978)
0808 16:59:34.497810 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.497845 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:37387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 16:59:34.498276 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.498132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:37387 (local address 127.0.0.1:37992)
0808 16:59:34.498193 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.498225 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:37387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 16:59:34.498413 28742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.125:37387: Network error: Client connection negotiation failed: client connection to 127.8.14.125:37387: connect: Connection refused (error 111)
I20260808 16:59:34.498484 28772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:34.498675 28750 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:34.498769 28750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-1/wal
uuid: "33e5574881eb43de8fca65edf5c4e74d"
format_stamp: "Formatted at 2026-08-08 16:59:34 on dist-test-slave-1nl6"
server_key: "0276d2cd9ccae16081fd705b5cc04d60"
server_key_iv: "1f4ea0c58eb0a18b9349d4875e032710"
server_key_version: "encryptionkey@0"
I20260808 16:59:34.498842 28750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:34.520707 28750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:34.522511 28750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:34.522675 28750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:34.526476 28750 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.125:37387
I20260808 16:59:34.526533 28853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.125:37387 every 8 connection(s)
I20260808 16:59:34.526880 28750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-1/data/info.pb
I20260808 16:59:34.527747 28854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:34.529222 28854 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:59:34.534426 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 28750
Aug 08 16:59:34 dist-test-slave-1nl6 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.125@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:34.534534 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-1/wal/instance
I20260808 16:59:34.534752 8249 external_mini_cluster.cc:1428] Setting key 285cf8e7b6e0cb4aabd75a7176ea674a
I20260808 16:59:34.538200 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.529893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55026 (local address 127.8.14.126:44133)
0808 16:59:34.529961 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:34.529963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:34.530155 (+ 192us) server_negotiation.cc:407] Connection header received
0808 16:59:34.530604 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:34.530604 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:34.530617 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:34.530674 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:34.531263 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.531742 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.532667 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.532868 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.532924 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.535091 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:34.535093 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:34.535093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:34.535100 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:34.536589 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.537113 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.537114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.537114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.537168 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.537450 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.537450 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.537450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.537550 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:34.537600 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:34.537954 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 16:59:34.538063 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 16:59:34.538210 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.529870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44133 (local address 127.0.0.1:55026)
0808 16:59:34.530031 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.530072 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:34.530525 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:34.530737 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:34.530740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:34.530995 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:34.531179 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.531183 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.531822 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.531823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.532593 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.532595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.532944 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.532945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.533040 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.533463 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:34.533471 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:34.535002 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:34.536672 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.536680 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.536682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.537040 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.537237 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.537238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.537239 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.537382 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.537617 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:34.537624 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:34.537789 (+ 165us) client_negotiation.cc:769] Sending connection context
0808 16:59:34.537908 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 16:59:34.538060 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260808 16:59:34.538755 28854 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 16:59:34 dist-test-slave-1nl6 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.125@KRBTEST.COM for kudu/127.8.14.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.14.124@KRBTEST.COM; defaulting to no policy
I20260808 16:59:34.546417 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.539037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:37387 (local address 127.0.0.1:37998)
0808 16:59:34.539092 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.539104 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:34.539148 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:34.539371 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:34.539372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:34.539552 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:34.539612 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.539614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.540493 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.540495 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.541367 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.541370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.541845 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.541846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.541925 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.542258 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:34.542263 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:34.543567 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:34.545371 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.545372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.545373 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.545536 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.545759 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.545759 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.545760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.545826 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.546109 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:34.546110 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:34.546147 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 16:59:34.546214 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 16:59:34.546309 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 16:59:34.546516 28858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.539076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37998 (local address 127.8.14.125:37387)
0808 16:59:34.539225 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:34.539230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:34.539251 (+ 21us) server_negotiation.cc:407] Connection header received
0808 16:59:34.539271 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:34.539272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:34.539282 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:34.539345 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:34.539690 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.540400 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.541487 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.541750 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.541807 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.543668 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:34.543676 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:34.543676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:34.543682 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:34.545257 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.545625 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.545628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.545628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.545689 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.545888 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.545889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.545889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.546030 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:34.546117 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:34.546249 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 16:59:34.546381 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":122,"thread_start_us":98,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.124@KRBTEST.COM" created.
I20260808 16:59:34.547188 28854 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:59:34.548610 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.548475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:40661 (local address 127.0.0.1:37610)
0808 16:59:34.548536 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.548580 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:40661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 16:59:34.548753 28778 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.14.124:40661: connect: Connection refused (error 111)
I20260808 16:59:34.549172 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.549080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:40661 (local address 127.0.0.1:37616)
0808 16:59:34.549128 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.549151 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:40661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 16:59:34.549281 28854 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.124:40661: Network error: Client connection negotiation failed: client connection to 127.8.14.124:40661: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.124.keytab.
Entry for principal kudu/127.8.14.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.124.keytab.
I20260808 16:59:34.561162 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.124:40661
--webserver_interface=127.8.14.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.124.keytab
--principal=kudu/127.8.14.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:44133,127.8.14.125:37387,127.8.14.124:40661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:59:34.589643 28742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } attempt: 3
Aug 08 16:59:34 dist-test-slave-1nl6 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for kudu/127.8.14.125@KRBTEST.COM
I20260808 16:59:34.596971 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.590020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:37387 (local address 127.0.0.1:38004)
0808 16:59:34.590081 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.590098 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:34.590161 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:34.590383 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:34.590384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:34.590535 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:34.590600 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.590602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.591558 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.591558 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.592044 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.592045 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.592581 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.592582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.592640 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.593100 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:34.593113 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:34.594270 (+ 1157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:34.596065 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.596066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.596066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.596228 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.596441 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.596442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.596442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.596493 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.596742 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:34.596743 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:34.596780 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 16:59:34.596835 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 16:59:34.596905 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 16:59:34.597043 28858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.590087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38004 (local address 127.8.14.125:37387)
0808 16:59:34.590152 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:34.590154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:34.590165 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:59:34.590263 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:34.590265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:34.590280 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:34.590366 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:34.590684 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.591481 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.592147 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.592501 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.592578 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.594354 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:34.594355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:34.594355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:34.594363 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:34.595978 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.596309 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.596310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.596310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.596373 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.596553 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.596554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.596554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.596681 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:34.596748 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:34.596874 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 16:59:34.596952 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 16:59:34.597491 28742 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:59:34.597853 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.597755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:40661 (local address 127.0.0.1:37620)
0808 16:59:34.597807 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.597835 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:40661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 16:59:34.598179 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.598091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:40661 (local address 127.0.0.1:37634)
0808 16:59:34.598133 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.598163 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:40661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 16:59:34.598263 28742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.124:40661: Network error: Client connection negotiation failed: client connection to 127.8.14.124:40661: connect: Connection refused (error 111)
I20260808 16:59:34.598498 28854 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } attempt: 1
I20260808 16:59:34.598801 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.598702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:40661 (local address 127.0.0.1:37638)
0808 16:59:34.598751 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.598776 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:40661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 16:59:34.599110 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.599018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:40661 (local address 127.0.0.1:37654)
0808 16:59:34.599067 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.599090 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:40661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 16:59:34.599203 28854 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.124:40661: Network error: Client connection negotiation failed: client connection to 127.8.14.124:40661: connect: Connection refused (error 111)
I20260808 16:59:34.629480 28742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } attempt: 1
I20260808 16:59:34.629917 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.629792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:40661 (local address 127.0.0.1:37658)
0808 16:59:34.629855 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.629891 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:40661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 16:59:34.630259 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.630162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:40661 (local address 127.0.0.1:37662)
0808 16:59:34.630208 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.630239 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:40661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 16:59:34.630354 28742 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.124:40661: Network error: Client connection negotiation failed: client connection to 127.8.14.124:40661: connect: Connection refused (error 111)
W20260808 16:59:34.645897 28862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:34.646090 28862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:34.646111 28862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:34.647294 28862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:59:34.647336 28862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:34.647351 28862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:34.647394 28862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:59:34.647413 28862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:59:34.648885 28862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:44133,127.8.14.125:37387,127.8.14.124:40661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.124.keytab
--principal=kudu/127.8.14.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.124:40661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.14.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:34.649112 28862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:34.649334 28862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:59:34.654258 28862 server_base.cc:1034] running on GCE node
W20260808 16:59:34.654351 28873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:34.654222 28875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:34.654241 28869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:34.654724 28862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:34.655076 28862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:34.656241 28862 hybrid_clock.cc:648] HybridClock initialized: now 1786208374656224 us; error 46 us; skew 500 ppm
Aug 08 16:59:34 dist-test-slave-1nl6 krb5kdc[28625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:34.658739 28862 init.cc:377] Logged in from keytab as kudu/127.8.14.124@KRBTEST.COM (short username kudu)
I20260808 16:59:34.659189 28862 webserver.cc:460] Webserver started at http://127.8.14.124:37341/ using document root <none> and password file <none>
I20260808 16:59:34.659385 28862 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:34.659426 28862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:34.659511 28862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:59:34.660480 28862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-2/data/instance:
uuid: "b2665248d03244be98266b2b2255bd70"
format_stamp: "Formatted at 2026-08-08 16:59:34 on dist-test-slave-1nl6"
server_key: "2e988e75a68cf93f4c54fbaa0c497700"
server_key_iv: "13683e5609bc6032fa1dd16b26bfe38f"
server_key_version: "encryptionkey@0"
I20260808 16:59:34.660789 28862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-2/wal/instance:
uuid: "b2665248d03244be98266b2b2255bd70"
format_stamp: "Formatted at 2026-08-08 16:59:34 on dist-test-slave-1nl6"
server_key: "2e988e75a68cf93f4c54fbaa0c497700"
server_key_iv: "13683e5609bc6032fa1dd16b26bfe38f"
server_key_version: "encryptionkey@0"
I20260808 16:59:34.662021 28862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:59:34.662683 28882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:34.662834 28862 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:59:34.662964 28862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-2/wal
uuid: "b2665248d03244be98266b2b2255bd70"
format_stamp: "Formatted at 2026-08-08 16:59:34 on dist-test-slave-1nl6"
server_key: "2e988e75a68cf93f4c54fbaa0c497700"
server_key_iv: "13683e5609bc6032fa1dd16b26bfe38f"
server_key_version: "encryptionkey@0"
I20260808 16:59:34.663033 28862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:34.667333 28854 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } attempt: 2
I20260808 16:59:34.667701 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.667576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:40661 (local address 127.0.0.1:37676)
0808 16:59:34.667635 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.667671 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:40661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 16:59:34.668082 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.668003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:40661 (local address 127.0.0.1:37684)
0808 16:59:34.668042 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.668066 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:40661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 16:59:34.668188 28854 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.124:40661: Network error: Client connection negotiation failed: client connection to 127.8.14.124:40661: connect: Connection refused (error 111)
I20260808 16:59:34.688964 28862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:34.691349 28862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:34.691462 28862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:34.695398 28862 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.124:40661
I20260808 16:59:34.695467 28962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.124:40661 every 8 connection(s)
I20260808 16:59:34.695799 28862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-2/data/info.pb
I20260808 16:59:34.696110 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 28862
I20260808 16:59:34.696264 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786208190438830-8249-0/minicluster-data/master-2/wal/instance
I20260808 16:59:34.696504 8249 external_mini_cluster.cc:1428] Setting key 04b2a45f8ca6d315667ed18026635d2a
I20260808 16:59:34.696913 28966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:34.699285 28966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 16:59:34 dist-test-slave-1nl6 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.124@KRBTEST.COM
Aug 08 16:59:34 dist-test-slave-1nl6 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.124@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:34.705494 28742 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } attempt: 2
I20260808 16:59:34.707291 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.699866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44133 (local address 127.0.0.1:55032)
0808 16:59:34.700055 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.700090 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:34.700158 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:34.700319 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:34.700321 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:34.700544 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:34.700605 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.700606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.701304 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.701305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.701886 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.701889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.702304 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.702305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.702404 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.702869 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:34.702877 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:34.704312 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:34.706014 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.706021 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.706023 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.706291 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.706528 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.706528 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.706529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.706601 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.706945 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:34.706952 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:34.707008 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 16:59:34.707092 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 16:59:34.707193 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":106,"threads_started":1}
I20260808 16:59:34.707237 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.697013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37700 (local address 127.8.14.124:40661)
0808 16:59:34.697205 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:34.697220 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:34.697245 (+ 25us) server_negotiation.cc:407] Connection header received
0808 16:59:34.697662 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:34.697667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:34.697711 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:34.697822 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:34.698585 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.699506 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.700494 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.700906 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.701029 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.703462 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:34.703470 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:34.703471 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:34.703478 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:34.705356 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.705797 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.705800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.705801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.705903 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.706151 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.706152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.706153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.706483 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:34.706606 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:34.706880 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 16:59:34.707039 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":101,"threads_started":1}
I20260808 16:59:34.707484 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.699963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55032 (local address 127.8.14.126:44133)
0808 16:59:34.700032 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:34.700035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:34.700161 (+ 126us) server_negotiation.cc:407] Connection header received
0808 16:59:34.700219 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:34.700220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:34.700230 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:34.700282 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:34.700699 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.701201 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.701978 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.702226 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.702290 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.704405 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:34.704406 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:34.704407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:34.704415 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:34.705943 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.706380 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.706382 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.706382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.706441 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.706676 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.706677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.706678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.706828 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:34.706893 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:34.707287 (+ 394us) server_negotiation.cc:299] Negotiation successful
0808 16:59:34.707383 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 16:59:34.708631 28966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 16:59:34 dist-test-slave-1nl6 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for kudu/127.8.14.124@KRBTEST.COM
Aug 08 16:59:34 dist-test-slave-1nl6 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.124@KRBTEST.COM for kudu/127.8.14.125@KRBTEST.COM
Aug 08 16:59:34 dist-test-slave-1nl6 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:34.714582 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.705772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37702 (local address 127.8.14.124:40661)
0808 16:59:34.705926 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:34.705928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:34.706096 (+ 168us) server_negotiation.cc:407] Connection header received
0808 16:59:34.706142 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:34.706143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:34.706154 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:34.706224 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:34.706668 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.707344 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.708406 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.709122 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.709220 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.711003 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:34.711004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:34.711005 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:34.711012 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:34.713048 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.713416 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.713417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.713417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.713474 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.713995 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.713996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.713996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.714130 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:34.714224 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:34.714391 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 16:59:34.714466 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260808 16:59:34.714609 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.705730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:40661 (local address 127.0.0.1:37702)
0808 16:59:34.706035 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.706045 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:34.706087 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:34.706250 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:34.706251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:34.706402 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:34.706452 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.706454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.707452 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.707453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.708175 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.708178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.709222 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.709223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.709280 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.709688 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:34.709693 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:34.710928 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:34.713129 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.713130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.713131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.713341 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.713544 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.713545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.713546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.713902 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.714224 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:34.714226 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:34.714270 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 16:59:34.714366 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 16:59:34.714509 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291}
I20260808 16:59:34.716472 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.708873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:37387 (local address 127.0.0.1:38014)
0808 16:59:34.709004 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.709015 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:34.709068 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:34.709234 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:34.709235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:34.709370 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:34.709438 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.709440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.710297 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.710298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.710999 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.711001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.711423 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.711424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.711492 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.711822 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:34.711827 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:34.713250 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:34.715451 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.715453 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.715453 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.715666 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.715889 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.715890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.715891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.715959 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.716243 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:34.716244 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:34.716285 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 16:59:34.716354 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 16:59:34.716419 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260808 16:59:34.716619 28858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.708948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38014 (local address 127.8.14.125:37387)
0808 16:59:34.709009 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:34.709011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:34.709055 (+ 44us) server_negotiation.cc:407] Connection header received
0808 16:59:34.709121 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:34.709122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:34.709132 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:34.709185 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:34.709530 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.710203 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.711067 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.711352 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.711412 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.713346 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:34.713348 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:34.713349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:34.713356 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:34.715368 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.715745 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.715746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.715746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.715808 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.716035 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.716036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.716036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.716163 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:34.716234 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:34.716413 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 16:59:34.716511 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 16:59:34.717084 28742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1: Bootstrap starting.
I20260808 16:59:34.717198 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.708445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55038 (local address 127.8.14.126:44133)
0808 16:59:34.708482 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:34.708483 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:34.708494 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:59:34.708561 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:34.708562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:34.708570 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:34.708614 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:34.709177 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.709803 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.710729 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.711039 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.711097 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.713938 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:34.713939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:34.713939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:34.713947 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:34.715607 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.716009 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.716011 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.716011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.716061 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.716578 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.716579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.716579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.716708 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:34.716867 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:34.717025 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 16:59:34.717093 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:59:34.717905 28742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:34.718212 28742 log.cc:824] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:34.718845 28966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } has no permanent_uuid. Determining permanent_uuid...
I20260808 16:59:34.719098 28742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1: No bootstrap required, opened a new log
I20260808 16:59:34.720916 28742 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } }
I20260808 16:59:34.721079 28742 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:34.721112 28742 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0bfb5a1b6374fbcbf323da5878159e1, State: Initialized, Role: FOLLOWER
I20260808 16:59:34.721261 28742 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } }
I20260808 16:59:34.721597 28742 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1 [sys.catalog]: configured and running, proceeding with master startup.
Aug 08 16:59:34 dist-test-slave-1nl6 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.125@KRBTEST.COM
Aug 08 16:59:34 dist-test-slave-1nl6 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.124@KRBTEST.COM for kudu/127.8.14.124@KRBTEST.COM
I20260808 16:59:34.723191 28973 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } } }
W20260808 16:59:34.723595 28987 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 16:59:34.724074 28973 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:59:34.726747 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.719105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:40661 (local address 127.0.0.1:37712)
0808 16:59:34.719157 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.719168 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:34.719204 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:34.719345 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:34.719345 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:34.719480 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:34.719506 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.719507 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.720366 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.720367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.720887 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.720890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.721411 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.721413 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.721467 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.721904 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:34.721908 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:34.723814 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:34.725710 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.725711 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.725711 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.725918 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.726100 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.726100 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.726101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.726152 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.726433 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:34.726434 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:34.726472 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 16:59:34.726537 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 16:59:34.726638 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 16:59:34.727010 28858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.717806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38016 (local address 127.8.14.125:37387)
0808 16:59:34.717840 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:34.717842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:34.717878 (+ 36us) server_negotiation.cc:407] Connection header received
0808 16:59:34.717900 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:34.717901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:34.717910 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:34.717969 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:34.718379 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.718987 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.719836 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.720033 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.720106 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.723071 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:34.723073 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:34.723073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:34.723080 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:34.724937 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.725407 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.725408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.725408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.725464 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.725673 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.725674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.725675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.725798 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:34.726760 (+ 962us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:34.726804 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 16:59:34.726889 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:59:34.727373 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.719172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37712 (local address 127.8.14.124:40661)
0808 16:59:34.719200 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:34.719201 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:34.719210 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:59:34.719254 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:34.719255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:34.719266 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:34.719339 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:34.719586 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.720273 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.720975 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.721318 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.721597 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.723880 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:34.723881 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:34.723882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:34.723888 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:34.725636 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.725989 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.725990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.725990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.726037 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.726215 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.726216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.726216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.726343 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:34.726409 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:34.727266 (+ 857us) server_negotiation.cc:299] Negotiation successful
0808 16:59:34.727326 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260808 16:59:34.725615 28645 master_runner.cc:186] Error getting master registration for 127.8.14.125:37387: OK, instance_id { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" instance_seqno: 1786208374525416 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 16:59:34.728587 28645 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 16:59:34.724089 28987 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:59:34.729945 28966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70: Bootstrap starting.
I20260808 16:59:34.730055 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:59:34.730547 28966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:34.730832 28966 log.cc:824] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:34.731731 28966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70: No bootstrap required, opened a new log
I20260808 16:59:34.733460 28966 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } }
I20260808 16:59:34.733629 28966 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:34.733681 28966 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2665248d03244be98266b2b2255bd70, State: Initialized, Role: FOLLOWER
I20260808 16:59:34.733789 28966 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } }
I20260808 16:59:34.734021 29002 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } } }
I20260808 16:59:34.734071 28966 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:59:34.734089 29002 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 16:59:34.736294 29024 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:59:34.736920 29024 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:59:34.739892 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.731636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55044 (local address 127.8.14.126:44133)
0808 16:59:34.731676 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:34.731677 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:34.731794 (+ 117us) server_negotiation.cc:407] Connection header received
0808 16:59:34.731906 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:34.731908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:34.731920 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:34.731984 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:34.732717 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.733399 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.734535 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.734849 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.734944 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.736520 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:34.736521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:34.736522 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:34.736530 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:34.738168 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.738547 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.738548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.738548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.738615 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.738831 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.738832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.738832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.739000 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:34.739088 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:34.739733 (+ 645us) server_negotiation.cc:299] Negotiation successful
0808 16:59:34.739802 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 16:59:34.738811 28862 master_runner.cc:186] Error getting master registration for 127.8.14.125:37387: OK, instance_id { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" instance_seqno: 1786208374525416 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 16:59:34.741061 28862 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 16:59:34.746974 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.735625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37720 (local address 127.8.14.124:40661)
0808 16:59:34.735656 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:34.735658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:34.735893 (+ 235us) server_negotiation.cc:407] Connection header received
0808 16:59:34.735997 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:34.735997 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:34.736008 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:34.736038 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:34.736779 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.737478 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.738498 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.738764 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.738805 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.744026 (+ 5221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:34.744027 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:34.744028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:34.744034 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:34.745632 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.746045 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.746047 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.746047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.746103 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.746317 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.746317 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.746318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.746443 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:34.746512 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:34.746736 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 16:59:34.746838 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 16:59:34.751255 28858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.743300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38024 (local address 127.8.14.125:37387)
0808 16:59:34.743341 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:34.743343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:34.743516 (+ 173us) server_negotiation.cc:407] Connection header received
0808 16:59:34.743540 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:34.743540 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:34.743550 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:34.743592 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:34.744581 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.745579 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.746598 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.746929 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.746989 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.748369 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:34.748370 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:34.748370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:34.748378 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:34.750007 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.750408 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.750409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.750410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.750471 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.750687 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.750688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.750688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.750823 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:34.750973 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:34.751085 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 16:59:34.751163 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:59:34.759343 28854 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } attempt: 3
Aug 08 16:59:34 dist-test-slave-1nl6 krb5kdc[28625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208374, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.125@KRBTEST.COM for kudu/127.8.14.124@KRBTEST.COM
I20260808 16:59:34.769255 28856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.759586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:40661 (local address 127.0.0.1:37722)
0808 16:59:34.759634 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:34.759644 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:34.759677 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:34.759838 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:34.759839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:34.760001 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:34.760042 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.760044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.760915 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.760916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.761701 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:34.761703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.764117 (+ 2414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.764118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:34.764189 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.764649 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:34.764655 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:34.766240 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:34.768163 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.768164 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.768165 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.768397 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.768612 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:34.768613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:34.768614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:34.768679 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:34.768960 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:34.768961 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:34.769003 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 16:59:34.769064 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 16:59:34.769156 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 16:59:34.769521 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:34.759663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37722 (local address 127.8.14.124:40661)
0808 16:59:34.759700 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:34.759702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:34.759728 (+ 26us) server_negotiation.cc:407] Connection header received
0808 16:59:34.759748 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:34.759749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:34.759758 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:34.759812 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:34.760115 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.760826 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.763719 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:34.764025 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:34.764100 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:34.766330 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:34.766331 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:34.766331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:34.766339 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:34.768066 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.768474 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.768476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.768476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.768537 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:34.768746 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:34.768747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:34.768747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:34.768877 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:34.768933 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:34.769369 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 16:59:34.769439 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:59:34.771060 28854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d: Bootstrap starting.
I20260808 16:59:34.771837 28854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:34.772212 28854 log.cc:824] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:34.773110 28854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d: No bootstrap required, opened a new log
I20260808 16:59:34.775269 28854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } }
I20260808 16:59:34.775413 28854 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:34.775447 28854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33e5574881eb43de8fca65edf5c4e74d, State: Initialized, Role: FOLLOWER
I20260808 16:59:34.775729 28854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } }
I20260808 16:59:34.776333 29031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } } }
I20260808 16:59:34.776378 28854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:59:34.776468 29031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [sys.catalog]: This master's current role is: FOLLOWER
W20260808 16:59:34.778342 29048 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:59:34.778425 29048 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:59:34.780153 28750 master_runner.cc:263] No leader master found from master 33e5574881eb43de8fca65edf5c4e74d
I20260808 16:59:34.780210 28750 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 16:59:34.832582 29031 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:59:34.832728 29031 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } }
I20260808 16:59:34.832971 29031 leader_election.cc:288] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0bfb5a1b6374fbcbf323da5878159e1 (127.8.14.126:44133), b2665248d03244be98266b2b2255bd70 (127.8.14.124:40661)
I20260808 16:59:34.833415 28919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33e5574881eb43de8fca65edf5c4e74d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2665248d03244be98266b2b2255bd70" is_pre_election: true
I20260808 16:59:34.833572 28919 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33e5574881eb43de8fca65edf5c4e74d in term 0.
I20260808 16:59:34.833926 28778 leader_election.cc:302] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33e5574881eb43de8fca65edf5c4e74d, b2665248d03244be98266b2b2255bd70; no voters:
I20260808 16:59:34.834098 29031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:59:34.834158 29031 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:59:34.834183 29031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:34.835124 28706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33e5574881eb43de8fca65edf5c4e74d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" is_pre_election: true
I20260808 16:59:34.835279 28706 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33e5574881eb43de8fca65edf5c4e74d in term 0.
I20260808 16:59:34.835845 29031 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } }
I20260808 16:59:34.835968 29031 leader_election.cc:288] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [CANDIDATE]: Term 1 election: Requested vote from peers a0bfb5a1b6374fbcbf323da5878159e1 (127.8.14.126:44133), b2665248d03244be98266b2b2255bd70 (127.8.14.124:40661)
I20260808 16:59:34.836236 28919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33e5574881eb43de8fca65edf5c4e74d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2665248d03244be98266b2b2255bd70"
I20260808 16:59:34.836314 28919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:34.837172 28919 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33e5574881eb43de8fca65edf5c4e74d in term 1.
I20260808 16:59:34.837368 28778 leader_election.cc:302] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33e5574881eb43de8fca65edf5c4e74d, b2665248d03244be98266b2b2255bd70; no voters:
I20260808 16:59:34.837607 28706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33e5574881eb43de8fca65edf5c4e74d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0bfb5a1b6374fbcbf323da5878159e1"
I20260808 16:59:34.837697 28706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:34.838577 28706 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33e5574881eb43de8fca65edf5c4e74d in term 1.
I20260808 16:59:34.838943 29031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:59:34.839118 29031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [term 1 LEADER]: Becoming Leader. State: Replica: 33e5574881eb43de8fca65edf5c4e74d, State: Running, Role: LEADER
I20260808 16:59:34.839257 29031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } }
I20260808 16:59:34.839675 29031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33e5574881eb43de8fca65edf5c4e74d. Latest consensus state: current_term: 1 leader_uuid: "33e5574881eb43de8fca65edf5c4e74d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } } }
I20260808 16:59:34.839771 29031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [sys.catalog]: This master's current role is: LEADER
I20260808 16:59:34.840057 29053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:59:34.840274 29053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:59:34.842329 28919 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70 [term 1 FOLLOWER]: Refusing update from remote peer 33e5574881eb43de8fca65edf5c4e74d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:59:34.842633 29031 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:59:34.843902 29002 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33e5574881eb43de8fca65edf5c4e74d. Latest consensus state: current_term: 1 leader_uuid: "33e5574881eb43de8fca65edf5c4e74d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } } }
I20260808 16:59:34.844043 29002 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:59:34.845448 28706 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1 [term 1 FOLLOWER]: Refusing update from remote peer 33e5574881eb43de8fca65edf5c4e74d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:59:34.845700 29031 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:59:34.846854 28973 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33e5574881eb43de8fca65edf5c4e74d. Latest consensus state: current_term: 1 leader_uuid: "33e5574881eb43de8fca65edf5c4e74d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } } }
I20260808 16:59:34.846969 28973 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:59:34.848351 29050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33e5574881eb43de8fca65edf5c4e74d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } } }
I20260808 16:59:34.848454 29050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [sys.catalog]: This master's current role is: LEADER
I20260808 16:59:34.848829 28973 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33e5574881eb43de8fca65edf5c4e74d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } } }
I20260808 16:59:34.848909 28973 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:59:34.849099 29054 mvcc.cc:205] Tried to move back new op lower bound from 7316309503351812096 to 7316309503342313472. Current Snapshot: MvccSnapshot[applied={T|T < 7316309503351812096}]
I20260808 16:59:34.849534 29058 mvcc.cc:205] Tried to move back new op lower bound from 7316309503351812096 to 7316309503342313472. Current Snapshot: MvccSnapshot[applied={T|T < 7316309503351812096}]
I20260808 16:59:34.849902 29031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33e5574881eb43de8fca65edf5c4e74d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } } }
I20260808 16:59:34.849985 29031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d [sys.catalog]: This master's current role is: LEADER
I20260808 16:59:34.850299 29002 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33e5574881eb43de8fca65edf5c4e74d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bfb5a1b6374fbcbf323da5878159e1" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44133 } } peers { permanent_uuid: "33e5574881eb43de8fca65edf5c4e74d" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 37387 } } peers { permanent_uuid: "b2665248d03244be98266b2b2255bd70" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 40661 } } }
I20260808 16:59:34.850387 29002 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 16:59:34.851011 29055 mvcc.cc:205] Tried to move back new op lower bound from 7316309503351812096 to 7316309503342313472. Current Snapshot: MvccSnapshot[applied={T|T < 7316309503351812096}]
I20260808 16:59:34.851487 29053 catalog_manager.cc:1347] Generated new cluster ID: eb60c86a5b5846bdb4d4d791fe1ad6bb
I20260808 16:59:34.851542 29053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:59:34.873860 29053 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:59:34.875667 29053 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:59:34.882010 29053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33e5574881eb43de8fca65edf5c4e74d: Generated new TSK 0
I20260808 16:59:34.882226 29053 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:59:35.729643 28987 catalog_manager.cc:1259] Loaded cluster ID: eb60c86a5b5846bdb4d4d791fe1ad6bb
I20260808 16:59:35.729744 28987 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1: loading cluster ID for follower catalog manager: success
I20260808 16:59:35.730569 28987 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1: acquiring CA information for follower catalog manager: success
I20260808 16:59:35.730973 28987 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a0bfb5a1b6374fbcbf323da5878159e1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 16:59:35.737658 29024 catalog_manager.cc:1259] Loaded cluster ID: eb60c86a5b5846bdb4d4d791fe1ad6bb
I20260808 16:59:35.737716 29024 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70: loading cluster ID for follower catalog manager: success
I20260808 16:59:35.738507 29024 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70: acquiring CA information for follower catalog manager: success
I20260808 16:59:35.738821 29024 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b2665248d03244be98266b2b2255bd70: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 16:59:35.938623 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 28645
I20260808 16:59:35.949869 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 28750
I20260808 16:59:35.959636 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 28862
2026-08-08T16:59:35Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3050 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:59:36 dist-test-slave-1nl6 krb5kdc[29072](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:59:36 dist-test-slave-1nl6 krb5kdc[29072](info): set up 2 sockets
Aug 08 16:59:36 dist-test-slave-1nl6 krb5kdc[29072](info): commencing operation
krb5kdc: starting...
W20260808 16:59:37.188803 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:59:37 dist-test-slave-1nl6 krb5kdc[29072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:59:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:59:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:59:37.307928 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:42097
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:33355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:42097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:59:37.388777 29095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:37.389005 29095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:37.389041 29095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:37.390146 29095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:59:37.390218 29095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:37.390240 29095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:37.390262 29095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:59:37.390300 29095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:59:37.391796 29095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:33355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:42097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:42097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:37.392073 29095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:37.392310 29095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:37.397184 29108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:37.397387 29095 server_base.cc:1034] running on GCE node
W20260808 16:59:37.397205 29106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:37.397197 29105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:37.397732 29095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:37.398036 29095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:37.399173 29095 hybrid_clock.cc:648] HybridClock initialized: now 1786208377399159 us; error 26 us; skew 500 ppm
Aug 08 16:59:37 dist-test-slave-1nl6 krb5kdc[29072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208377, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:37.401708 29095 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:59:37.402099 29095 webserver.cc:460] Webserver started at http://127.8.14.126:38171/ using document root <none> and password file <none>
I20260808 16:59:37.402323 29095 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:37.402364 29095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:37.402448 29095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:59:37.403481 29095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "44c74f18a5b741d08885276f500f3eb3"
format_stamp: "Formatted at 2026-08-08 16:59:37 on dist-test-slave-1nl6"
server_key: "0f198726037d70d801cdebc7db1f4e65"
server_key_iv: "e88a2409108087bd3ebd4ada6a88b6a8"
server_key_version: "encryptionkey@0"
I20260808 16:59:37.403816 29095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "44c74f18a5b741d08885276f500f3eb3"
format_stamp: "Formatted at 2026-08-08 16:59:37 on dist-test-slave-1nl6"
server_key: "0f198726037d70d801cdebc7db1f4e65"
server_key_iv: "e88a2409108087bd3ebd4ada6a88b6a8"
server_key_version: "encryptionkey@0"
I20260808 16:59:37.405049 29095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:59:37.405730 29122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:37.405910 29095 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:59:37.406020 29095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "44c74f18a5b741d08885276f500f3eb3"
format_stamp: "Formatted at 2026-08-08 16:59:37 on dist-test-slave-1nl6"
server_key: "0f198726037d70d801cdebc7db1f4e65"
server_key_iv: "e88a2409108087bd3ebd4ada6a88b6a8"
server_key_version: "encryptionkey@0"
I20260808 16:59:37.406096 29095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:37.418434 29095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:37.420668 29095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:37.420837 29095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:37.424942 29095 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:42097
I20260808 16:59:37.424975 29200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:42097 every 8 connection(s)
I20260808 16:59:37.425304 29095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:59:37.426234 29201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:37.428567 29201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3: Bootstrap starting.
I20260808 16:59:37.429162 29201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:37.429464 29201 log.cc:824] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:37.430194 29201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3: No bootstrap required, opened a new log
I20260808 16:59:37.431804 29201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c74f18a5b741d08885276f500f3eb3" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42097 } }
I20260808 16:59:37.431936 29201 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:37.432000 29201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44c74f18a5b741d08885276f500f3eb3, State: Initialized, Role: FOLLOWER
I20260808 16:59:37.432158 29201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c74f18a5b741d08885276f500f3eb3" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42097 } }
I20260808 16:59:37.432224 29201 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:59:37.432246 29201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:59:37.432322 29201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:37.432677 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 29095
I20260808 16:59:37.432778 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:59:37.432946 29201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c74f18a5b741d08885276f500f3eb3" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42097 } }
I20260808 16:59:37.433014 8249 external_mini_cluster.cc:1428] Setting key 2533ad0c29575af22be7c1edf135644f
I20260808 16:59:37.433055 29201 leader_election.cc:302] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44c74f18a5b741d08885276f500f3eb3; no voters:
I20260808 16:59:37.433212 29201 leader_election.cc:288] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:59:37.433238 29208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:59:37.433370 29208 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 44c74f18a5b741d08885276f500f3eb3, State: Running, Role: LEADER
I20260808 16:59:37.433470 29201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:59:37.433452 29208 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c74f18a5b741d08885276f500f3eb3" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42097 } }
I20260808 16:59:37.433794 29208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44c74f18a5b741d08885276f500f3eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c74f18a5b741d08885276f500f3eb3" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42097 } } }
I20260808 16:59:37.433873 29208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [sys.catalog]: This master's current role is: LEADER
I20260808 16:59:37.434010 29208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44c74f18a5b741d08885276f500f3eb3. Latest consensus state: current_term: 1 leader_uuid: "44c74f18a5b741d08885276f500f3eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c74f18a5b741d08885276f500f3eb3" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42097 } } }
I20260808 16:59:37.434082 29208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3 [sys.catalog]: This master's current role is: LEADER
I20260808 16:59:37.434862 29219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:59:37.435477 29219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:59:37.438416 29219 catalog_manager.cc:1347] Generated new cluster ID: 5319a6dbcfaa4317a2becf2b6a975dd1
I20260808 16:59:37.438475 29219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:59:37 dist-test-slave-1nl6 krb5kdc[29072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:37.443542 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:37.433486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37244 (local address 127.8.14.126:42097)
0808 16:59:37.433773 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:37.433784 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:37.433835 (+ 51us) server_negotiation.cc:407] Connection header received
0808 16:59:37.434197 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:37.434205 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:37.434267 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:37.434449 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:37.435261 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:37.436136 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:37.437177 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:37.437781 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:37.437906 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:37.440288 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:37.440292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:37.440292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:37.440303 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:37.441963 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:37.442360 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:37.442363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:37.442363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:37.442463 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:37.442685 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:37.442686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:37.442686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:37.442979 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:37.443146 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:37.443244 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 16:59:37.443385 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":79,"threads_started":1}
I20260808 16:59:37.444474 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 16:59:37.453032 29219 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:59:37.453538 29219 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:59:37.462819 29219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44c74f18a5b741d08885276f500f3eb3: Generated new TSK 0
I20260808 16:59:37.463032 29219 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 16:59:37.498488 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 29095
2026-08-08T16:59:37Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1537 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:59:37 dist-test-slave-1nl6 krb5kdc[29238](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:59:37 dist-test-slave-1nl6 krb5kdc[29238](info): set up 2 sockets
Aug 08 16:59:37 dist-test-slave-1nl6 krb5kdc[29238](info): commencing operation
krb5kdc: starting...
W20260808 16:59:38.725548 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:59:38 dist-test-slave-1nl6 krb5kdc[29238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:59:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:59:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:59:38.848929 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:45175
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:45175
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:59:38.930508 29258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:38.930727 29258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:38.930763 29258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:38.931890 29258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:59:38.931951 29258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:38.931972 29258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:38.931993 29258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:59:38.932020 29258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:59:38.933499 29258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:45175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:45175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:38.933759 29258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:38.933969 29258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 16:59:38.938721 29258 server_base.cc:1034] running on GCE node
W20260808 16:59:38.938686 29272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:38.938700 29275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:38.938701 29268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:38.939217 29258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:38.939491 29258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:38.940650 29258 hybrid_clock.cc:648] HybridClock initialized: now 1786208378940643 us; error 35 us; skew 500 ppm
Aug 08 16:59:38 dist-test-slave-1nl6 krb5kdc[29238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208378, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:38.943051 29258 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:59:38.943464 29258 webserver.cc:460] Webserver started at http://127.8.14.126:40643/ using document root <none> and password file <none>
I20260808 16:59:38.943660 29258 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:38.943719 29258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:38.943806 29258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:59:38.944737 29258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "87f8e814f3314147bd30d9eb7d9382db"
format_stamp: "Formatted at 2026-08-08 16:59:38 on dist-test-slave-1nl6"
server_key: "8d52ae1c84779e3f89c02285eca8bab1"
server_key_iv: "71eb4ae72d0dca6c10b73a767b13ad41"
server_key_version: "encryptionkey@0"
I20260808 16:59:38.945037 29258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "87f8e814f3314147bd30d9eb7d9382db"
format_stamp: "Formatted at 2026-08-08 16:59:38 on dist-test-slave-1nl6"
server_key: "8d52ae1c84779e3f89c02285eca8bab1"
server_key_iv: "71eb4ae72d0dca6c10b73a767b13ad41"
server_key_version: "encryptionkey@0"
I20260808 16:59:38.946265 29258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:59:38.947000 29285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:38.947168 29258 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 16:59:38.947247 29258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "87f8e814f3314147bd30d9eb7d9382db"
format_stamp: "Formatted at 2026-08-08 16:59:38 on dist-test-slave-1nl6"
server_key: "8d52ae1c84779e3f89c02285eca8bab1"
server_key_iv: "71eb4ae72d0dca6c10b73a767b13ad41"
server_key_version: "encryptionkey@0"
I20260808 16:59:38.947319 29258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:38.959661 29258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:38.961864 29258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:38.962019 29258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:38.965907 29258 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:45175
I20260808 16:59:38.965982 29362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:45175 every 8 connection(s)
I20260808 16:59:38.966297 29258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:59:38.967300 29365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:38.969493 29365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db: Bootstrap starting.
I20260808 16:59:38.970026 29365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:38.970257 29365 log.cc:824] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:38.970950 29365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db: No bootstrap required, opened a new log
I20260808 16:59:38.972507 29365 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } }
I20260808 16:59:38.972640 29365 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:38.972662 29365 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87f8e814f3314147bd30d9eb7d9382db, State: Initialized, Role: FOLLOWER
I20260808 16:59:38.972764 29365 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } }
I20260808 16:59:38.972816 29365 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:59:38.972836 29365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:59:38.972867 29365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:38.973541 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 29258
I20260808 16:59:38.973656 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:59:38.973748 29365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } }
I20260808 16:59:38.973867 29365 leader_election.cc:302] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87f8e814f3314147bd30d9eb7d9382db; no voters:
I20260808 16:59:38.973901 8249 external_mini_cluster.cc:1428] Setting key a7788436ae5db415a3ea08afc682909b
I20260808 16:59:38.974042 29365 leader_election.cc:288] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:59:38.974085 29371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:59:38.974304 29371 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 1 LEADER]: Becoming Leader. State: Replica: 87f8e814f3314147bd30d9eb7d9382db, State: Running, Role: LEADER
I20260808 16:59:38.974421 29365 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:59:38.974471 29371 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } }
I20260808 16:59:38.974803 29371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87f8e814f3314147bd30d9eb7d9382db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } } }
I20260808 16:59:38.974946 29371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [sys.catalog]: This master's current role is: LEADER
I20260808 16:59:38.974864 29373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87f8e814f3314147bd30d9eb7d9382db. Latest consensus state: current_term: 1 leader_uuid: "87f8e814f3314147bd30d9eb7d9382db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } } }
I20260808 16:59:38.975050 29373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [sys.catalog]: This master's current role is: LEADER
I20260808 16:59:38.975274 29391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:59:38.975893 29391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:59:38.977427 29391 catalog_manager.cc:1347] Generated new cluster ID: 81bc815e5acd466597eec2d3e7dd71ad
I20260808 16:59:38.977469 29391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 16:59:38 dist-test-slave-1nl6 krb5kdc[29238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:38.985049 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:38.974359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37680 (local address 127.8.14.126:45175)
0808 16:59:38.974525 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:38.974534 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:38.974744 (+ 210us) server_negotiation.cc:407] Connection header received
0808 16:59:38.974874 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:38.974882 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:38.974973 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:38.975971 (+ 998us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:38.976077 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:38.977169 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:38.978004 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:38.978380 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:38.978494 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:38.981180 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:38.981184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:38.981185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:38.981197 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:38.983226 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:38.983632 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:38.983635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:38.983636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:38.983766 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:38.983957 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:38.983958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:38.983959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:38.984281 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:38.984462 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:38.984529 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 16:59:38.984676 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260808 16:59:38.992844 29391 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:59:38.993304 29391 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:59:38.995294 29391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db: Generated new TSK 0
I20260808 16:59:38.995453 29391 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 16:59:39.029877 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:45175
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:59:39.112190 29411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:39.112385 29411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:39.112414 29411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:39.113523 29411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:39.113564 29411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:39.113590 29411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:59:39.115141 29411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:45175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:39.115379 29411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:39.115618 29411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:39.120545 29419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:39.120570 29421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:39.120657 29411 server_base.cc:1034] running on GCE node
W20260808 16:59:39.120744 29424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:39.120951 29411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:39.121181 29411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:39.122334 29411 hybrid_clock.cc:648] HybridClock initialized: now 1786208379122320 us; error 28 us; skew 500 ppm
Aug 08 16:59:39 dist-test-slave-1nl6 krb5kdc[29238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208379, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:39.124723 29411 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:59:39.125145 29411 webserver.cc:460] Webserver started at http://127.8.14.65:37849/ using document root <none> and password file <none>
I20260808 16:59:39.125427 29411 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:39.125475 29411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:39.125623 29411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:59:39.126731 29411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "bb8fbeac543346de8fd0593d3f7b59dd"
format_stamp: "Formatted at 2026-08-08 16:59:39 on dist-test-slave-1nl6"
server_key: "9905d26121efbb62cfbced5d405290b7"
server_key_iv: "802972def4b9dc1cee0f9ac72f71b0aa"
server_key_version: "encryptionkey@0"
I20260808 16:59:39.127118 29411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "bb8fbeac543346de8fd0593d3f7b59dd"
format_stamp: "Formatted at 2026-08-08 16:59:39 on dist-test-slave-1nl6"
server_key: "9905d26121efbb62cfbced5d405290b7"
server_key_iv: "802972def4b9dc1cee0f9ac72f71b0aa"
server_key_version: "encryptionkey@0"
I20260808 16:59:39.128386 29411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:59:39.129067 29437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:39.129235 29411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:59:39.129338 29411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "bb8fbeac543346de8fd0593d3f7b59dd"
format_stamp: "Formatted at 2026-08-08 16:59:39 on dist-test-slave-1nl6"
server_key: "9905d26121efbb62cfbced5d405290b7"
server_key_iv: "802972def4b9dc1cee0f9ac72f71b0aa"
server_key_version: "encryptionkey@0"
I20260808 16:59:39.129415 29411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:39.146579 29411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:39.148377 29411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:39.148528 29411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:39.148744 29411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:39.149048 29411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:59:39.149101 29411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:39.149142 29411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:59:39.149170 29411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:39.155002 29411 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:33131
I20260808 16:59:39.155058 29604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:33131 every 8 connection(s)
I20260808 16:59:39.155380 29411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
Aug 08 16:59:39 dist-test-slave-1nl6 krb5kdc[29238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208379, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:39.164350 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.156052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:48421 (local address 127.8.14.126:45175)
0808 16:59:39.156094 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:39.156097 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:39.156268 (+ 171us) server_negotiation.cc:407] Connection header received
0808 16:59:39.156773 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:39.156774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:39.156791 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:39.156854 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:39.157750 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.158265 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.159331 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.159542 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.159616 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.161738 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:39.161740 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:39.161740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:39.161749 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:39.163020 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:39.163398 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:39.163399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:39.163400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:39.163462 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:39.163694 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:39.163695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:39.163696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:39.163829 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:39.163898 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:39.164146 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 16:59:39.164251 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 16:59:39.164460 29607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.156015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:48421)
0808 16:59:39.156152 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:39.156197 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:39.156669 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:39.156921 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:39.156925 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:39.157466 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:39.157642 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:39.157646 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.158368 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.158369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:39.159241 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:39.159243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.159638 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.159639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:39.159755 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.160261 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:39.160270 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:39.161645 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:39.163097 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:39.163105 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:39.163107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:39.163322 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:39.163531 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:39.163532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:39.163532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:39.163630 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:39.163903 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:39.163910 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:39.164030 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 16:59:39.164124 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 16:59:39.164262 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":64,"threads_started":1}
I20260808 16:59:39.165174 29605 heartbeater.cc:347] Connected to a master server at 127.8.14.126:45175
I20260808 16:59:39.165259 29605 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:39.165297 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 29411
I20260808 16:59:39.165400 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:59:39.165478 29605 heartbeater.cc:511] Master 127.8.14.126:45175 requested a full tablet report, sending...
I20260808 16:59:39.165623 8249 external_mini_cluster.cc:1428] Setting key b32ff84b0bc59148e596c7776a78ba9d
I20260808 16:59:39.166081 29310 ts_manager.cc:194] Registered new tserver with Master: bb8fbeac543346de8fd0593d3f7b59dd (127.8.14.65:33131)
I20260808 16:59:39.166725 29310 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:48421
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 16:59:39.189723 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:45175
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:59:39.275537 29610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:39.275763 29610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:39.275801 29610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:39.276904 29610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:39.276968 29610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:39.277011 29610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:59:39.278456 29610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:45175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:39.278728 29610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:39.279002 29610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:39.283921 29621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:39.283975 29610 server_base.cc:1034] running on GCE node
W20260808 16:59:39.283921 29627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:39.283941 29622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:39.284373 29610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:39.284622 29610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:39.285764 29610 hybrid_clock.cc:648] HybridClock initialized: now 1786208379285751 us; error 30 us; skew 500 ppm
Aug 08 16:59:39 dist-test-slave-1nl6 krb5kdc[29238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208379, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:39.288373 29610 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:59:39.288846 29610 webserver.cc:460] Webserver started at http://127.8.14.66:45707/ using document root <none> and password file <none>
I20260808 16:59:39.289086 29610 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:39.289152 29610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:39.289294 29610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:59:39.290293 29610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "e877162b46d64529850b1dc1a0bd79a2"
format_stamp: "Formatted at 2026-08-08 16:59:39 on dist-test-slave-1nl6"
server_key: "d266bbf42be5e7216cd2e10cae12e5d4"
server_key_iv: "823efe4ffdacad2f9314dd495b6f6b7d"
server_key_version: "encryptionkey@0"
I20260808 16:59:39.290644 29610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "e877162b46d64529850b1dc1a0bd79a2"
format_stamp: "Formatted at 2026-08-08 16:59:39 on dist-test-slave-1nl6"
server_key: "d266bbf42be5e7216cd2e10cae12e5d4"
server_key_iv: "823efe4ffdacad2f9314dd495b6f6b7d"
server_key_version: "encryptionkey@0"
I20260808 16:59:39.292039 29610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 16:59:39.292780 29634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:39.292959 29610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:59:39.293056 29610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "e877162b46d64529850b1dc1a0bd79a2"
format_stamp: "Formatted at 2026-08-08 16:59:39 on dist-test-slave-1nl6"
server_key: "d266bbf42be5e7216cd2e10cae12e5d4"
server_key_iv: "823efe4ffdacad2f9314dd495b6f6b7d"
server_key_version: "encryptionkey@0"
I20260808 16:59:39.293138 29610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:39.302160 29610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:39.304450 29610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:39.304605 29610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:39.304822 29610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:39.305114 29610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:59:39.305164 29610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:39.305205 29610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:59:39.305234 29610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:39.311085 29610 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:46293
I20260808 16:59:39.311136 29798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:46293 every 8 connection(s)
I20260808 16:59:39.311486 29610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:59:39.314687 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 29610
I20260808 16:59:39.314808 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:59:39.315037 8249 external_mini_cluster.cc:1428] Setting key f84c91de01cfcd0b46f8cb268438cffe
Aug 08 16:59:39 dist-test-slave-1nl6 krb5kdc[29238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208379, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:39.321125 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.312208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:54405 (local address 127.8.14.126:45175)
0808 16:59:39.312274 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:39.312277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:39.312496 (+ 219us) server_negotiation.cc:407] Connection header received
0808 16:59:39.312960 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:39.312961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:39.312971 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:39.313020 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:39.313897 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.314414 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.315603 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.315818 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.315903 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.318149 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:39.318151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:39.318151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:39.318160 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:39.319594 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:39.320140 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:39.320141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:39.320142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:39.320197 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:39.320500 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:39.320500 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:39.320501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:39.320602 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:39.320654 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:39.320987 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 16:59:39.321061 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 16:59:39.321426 29802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.312185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:54405)
0808 16:59:39.312382 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:39.312417 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:39.312877 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:39.313078 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:39.313081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:39.313629 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:39.313816 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:39.313822 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.314490 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.314491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:39.315505 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:39.315508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.315915 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.315916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:39.316054 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.316545 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:39.316554 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:39.318058 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:39.319676 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:39.319684 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:39.319686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:39.320057 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:39.320265 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:39.320266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:39.320266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:39.320425 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:39.320671 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:39.320678 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:39.320845 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 16:59:39.320968 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 16:59:39.321153 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":114,"threads_started":1}
I20260808 16:59:39.321913 29799 heartbeater.cc:347] Connected to a master server at 127.8.14.126:45175
I20260808 16:59:39.322023 29799 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:39.322198 29799 heartbeater.cc:511] Master 127.8.14.126:45175 requested a full tablet report, sending...
I20260808 16:59:39.322633 29310 ts_manager.cc:194] Registered new tserver with Master: e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66:46293)
I20260808 16:59:39.323158 29310 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:54405
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 16:59:39.341512 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:45175
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:59:39.423331 29806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:39.423558 29806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:39.423595 29806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:39.424712 29806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:39.424779 29806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:39.424830 29806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:59:39.426282 29806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:45175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:39.426540 29806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:39.426762 29806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:39.431645 29816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:39.431828 29806 server_base.cc:1034] running on GCE node
W20260808 16:59:39.431615 29825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:39.431603 29818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:39.432118 29806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:39.432394 29806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:39.433532 29806 hybrid_clock.cc:648] HybridClock initialized: now 1786208379433521 us; error 29 us; skew 500 ppm
Aug 08 16:59:39 dist-test-slave-1nl6 krb5kdc[29238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208379, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:39.440145 29806 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:59:39.440539 29806 webserver.cc:460] Webserver started at http://127.8.14.67:39041/ using document root <none> and password file <none>
I20260808 16:59:39.440718 29806 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:39.440758 29806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:39.440847 29806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:59:39.441823 29806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "62a5955432af4f1ebd619d258ec5e829"
format_stamp: "Formatted at 2026-08-08 16:59:39 on dist-test-slave-1nl6"
server_key: "0b155a473ddb646aee7bfcfef1d57eae"
server_key_iv: "483beff439602a9fd461293f03f7ba3e"
server_key_version: "encryptionkey@0"
I20260808 16:59:39.442147 29806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "62a5955432af4f1ebd619d258ec5e829"
format_stamp: "Formatted at 2026-08-08 16:59:39 on dist-test-slave-1nl6"
server_key: "0b155a473ddb646aee7bfcfef1d57eae"
server_key_iv: "483beff439602a9fd461293f03f7ba3e"
server_key_version: "encryptionkey@0"
I20260808 16:59:39.443468 29806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 16:59:39.444156 29834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:39.444326 29806 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:39.444407 29806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "62a5955432af4f1ebd619d258ec5e829"
format_stamp: "Formatted at 2026-08-08 16:59:39 on dist-test-slave-1nl6"
server_key: "0b155a473ddb646aee7bfcfef1d57eae"
server_key_iv: "483beff439602a9fd461293f03f7ba3e"
server_key_version: "encryptionkey@0"
I20260808 16:59:39.444461 29806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:39.454643 29806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:39.456421 29806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:39.456529 29806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:39.456699 29806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:39.456992 29806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:59:39.457026 29806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:39.457065 29806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:59:39.457091 29806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:39.462682 29806 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:35721
I20260808 16:59:39.462720 30006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:35721 every 8 connection(s)
I20260808 16:59:39.463076 29806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 16:59:39.466264 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 29806
I20260808 16:59:39.466357 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:59:39.466538 8249 external_mini_cluster.cc:1428] Setting key 213f706d17f14e40c451d6d4dbff5484
Aug 08 16:59:39 dist-test-slave-1nl6 krb5kdc[29238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208379, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:39.471827 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.463800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:54481 (local address 127.8.14.126:45175)
0808 16:59:39.463875 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:39.463877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:39.464031 (+ 154us) server_negotiation.cc:407] Connection header received
0808 16:59:39.464473 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:39.464474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:39.464485 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:39.464541 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:39.465395 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.465882 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.467149 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.467452 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.467518 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.469522 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:39.469523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:39.469524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:39.469531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:39.470666 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:39.471123 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:39.471124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:39.471124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:39.471175 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:39.471370 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:39.471370 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:39.471371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:39.471458 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:39.471503 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:39.471699 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 16:59:39.471757 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 16:59:39.472040 30010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.463743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:54481)
0808 16:59:39.463903 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:39.463942 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:39.464389 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:39.464589 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:39.464592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:39.465128 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:39.465306 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:39.465310 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.465959 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.465960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:39.467002 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:39.467005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.467535 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.467536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:39.467652 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.468143 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:39.468152 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:39.469441 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:39.470762 (+ 1321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:39.470769 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:39.470772 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:39.471053 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:39.471230 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:39.471231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:39.471231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:39.471329 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:39.471505 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:39.471509 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:39.471610 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 16:59:39.471686 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 16:59:39.471794 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260808 16:59:39.472501 30007 heartbeater.cc:347] Connected to a master server at 127.8.14.126:45175
I20260808 16:59:39.472596 30007 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:39.472756 30007 heartbeater.cc:511] Master 127.8.14.126:45175 requested a full tablet report, sending...
I20260808 16:59:39.473119 29310 ts_manager.cc:194] Registered new tserver with Master: 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721)
I20260808 16:59:39.473476 29310 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:54481
I20260808 16:59:39.478096 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:59:39.485759 29377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.479563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37696 (local address 127.8.14.126:45175)
0808 16:59:39.479614 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:39.479616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:39.479731 (+ 115us) server_negotiation.cc:407] Connection header received
0808 16:59:39.479774 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:39.479774 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:39.479782 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:39.479838 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:39.480513 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.481260 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.481971 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.482144 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.482194 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.483417 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:39.483418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:39.483418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:39.483424 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:39.484560 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:39.484911 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:39.484912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:39.484912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:39.484974 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:39.485175 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:39.485176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:39.485176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:39.485328 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:39.485433 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:39.485569 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 16:59:39.485655 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:59:39.487524 29310 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:59:39.489882 29310 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:59:39.497570 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.492288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:35721 (local address 127.0.0.1:48670)
0808 16:59:39.492815 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:39.492858 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:39.492927 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:39.493283 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:39.493287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:39.493291 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:39.493383 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:39.493386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.494742 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.494743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:39.496176 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:39.496180 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.497270 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.497271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:39.497359 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.497374 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 16:59:39.497440 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 16:59:39.497490 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":36,"thread_start_us":54,"threads_started":1}
I20260808 16:59:39.498005 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.492430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48670 (local address 127.8.14.67:35721)
0808 16:59:39.493082 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:39.493087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:39.493098 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:59:39.493139 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:39.493141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:39.493158 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:39.493250 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:39.493477 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.494621 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.496275 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.497108 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.497170 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.497235 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:39.497876 (+ 641us) server_negotiation.cc:299] Negotiation successful
0808 16:59:39.497919 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":109,"threads_started":1}
I20260808 16:59:39.498198 30025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.492298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:33131 (local address 127.0.0.1:33454)
0808 16:59:39.493251 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:39.493262 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:39.493322 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:39.493605 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:39.493606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:39.493608 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:39.493659 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:39.493661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.495180 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.495181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:39.496639 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:39.496642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.497965 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.497966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:39.498030 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.498034 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:59:39.498088 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 16:59:39.498132 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"thread_start_us":53,"threads_started":1}
I20260808 16:59:39.498474 30033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.492690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33454 (local address 127.8.14.65:33131)
0808 16:59:39.493204 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:39.493208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:39.493375 (+ 167us) server_negotiation.cc:407] Connection header received
0808 16:59:39.493419 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:39.493421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:39.493452 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:39.493525 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:39.493737 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.495078 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.496732 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.497815 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.497875 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.497931 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:39.498340 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 16:59:39.498391 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":95,"threads_started":1}
I20260808 16:59:39.499125 29902 tablet_service.cc:1467] Processing CreateTablet for tablet 79343eed25034ea39b87b0403aa74c8f (DEFAULT_TABLE table=test-table [id=75271de0384c4cb6aef45bbeca0cc0f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:59:39.499320 29508 tablet_service.cc:1467] Processing CreateTablet for tablet 79343eed25034ea39b87b0403aa74c8f (DEFAULT_TABLE table=test-table [id=75271de0384c4cb6aef45bbeca0cc0f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:59:39.499509 29902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79343eed25034ea39b87b0403aa74c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:39.499679 29508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79343eed25034ea39b87b0403aa74c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:39.502184 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.492480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:46293 (local address 127.0.0.1:60354)
0808 16:59:39.497738 (+ 5258us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:39.497747 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:39.497785 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:39.498154 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:39.498155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:39.498157 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:39.498192 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:39.498194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.499584 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.499585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:39.501030 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:39.501033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.501922 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.501923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:39.501993 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.501997 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:59:39.502062 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 16:59:39.502103 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5248,"thread_start_us":56,"threads_started":1}
I20260808 16:59:39.502221 30032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.492604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60354 (local address 127.8.14.66:46293)
0808 16:59:39.492823 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:39.492829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:39.497877 (+ 5048us) server_negotiation.cc:407] Connection header received
0808 16:59:39.497933 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:39.497936 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:39.497955 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:39.498039 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:39.498277 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.499417 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.501119 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.501824 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.501902 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.501953 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:39.502093 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 16:59:39.502141 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":90,"threads_started":1}
I20260808 16:59:39.502583 30036 tablet_bootstrap.cc:492] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829: Bootstrap starting.
I20260808 16:59:39.503089 29708 tablet_service.cc:1467] Processing CreateTablet for tablet 79343eed25034ea39b87b0403aa74c8f (DEFAULT_TABLE table=test-table [id=75271de0384c4cb6aef45bbeca0cc0f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:59:39.503438 29708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79343eed25034ea39b87b0403aa74c8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:39.503530 30036 tablet_bootstrap.cc:654] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:39.504148 30036 log.cc:824] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:39.505044 30036 tablet_bootstrap.cc:492] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829: No bootstrap required, opened a new log
I20260808 16:59:39.505131 30036 ts_tablet_manager.cc:1397] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 16:59:39.506240 30040 tablet_bootstrap.cc:492] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2: Bootstrap starting.
I20260808 16:59:39.506712 30038 tablet_bootstrap.cc:492] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: Bootstrap starting.
I20260808 16:59:39.507128 30040 tablet_bootstrap.cc:654] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:39.507547 30038 tablet_bootstrap.cc:654] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:39.507756 30040 log.cc:824] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:39.507702 30036 raft_consensus.cc:348] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } }
I20260808 16:59:39.508030 30036 raft_consensus.cc:374] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:39.508139 30036 raft_consensus.cc:729] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a5955432af4f1ebd619d258ec5e829, State: Initialized, Role: FOLLOWER
I20260808 16:59:39.508183 30038 log.cc:824] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:39.508352 30036 consensus_queue.cc:260] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } }
I20260808 16:59:39.508566 30040 tablet_bootstrap.cc:492] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2: No bootstrap required, opened a new log
I20260808 16:59:39.508644 30040 ts_tablet_manager.cc:1397] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:59:39.508677 30036 ts_tablet_manager.cc:1428] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 16:59:39.509330 30038 tablet_bootstrap.cc:492] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: No bootstrap required, opened a new log
I20260808 16:59:39.509405 30038 ts_tablet_manager.cc:1397] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 16:59:39.510176 30040 raft_consensus.cc:348] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } }
I20260808 16:59:39.510326 30040 raft_consensus.cc:374] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:39.510371 30040 raft_consensus.cc:729] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e877162b46d64529850b1dc1a0bd79a2, State: Initialized, Role: FOLLOWER
I20260808 16:59:39.510491 30040 consensus_queue.cc:260] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } }
I20260808 16:59:39.510777 30040 ts_tablet_manager.cc:1428] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:59:39.510790 29799 heartbeater.cc:503] Master 127.8.14.126:45175 was elected leader, sending a full tablet report...
I20260808 16:59:39.510972 30007 heartbeater.cc:503] Master 127.8.14.126:45175 was elected leader, sending a full tablet report...
I20260808 16:59:39.511579 30038 raft_consensus.cc:348] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } }
I20260808 16:59:39.511776 30038 raft_consensus.cc:374] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:39.511853 30038 raft_consensus.cc:729] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb8fbeac543346de8fd0593d3f7b59dd, State: Initialized, Role: FOLLOWER
I20260808 16:59:39.512009 30038 consensus_queue.cc:260] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } }
I20260808 16:59:39.513592 30038 ts_tablet_manager.cc:1428] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260808 16:59:39.513615 29605 heartbeater.cc:503] Master 127.8.14.126:45175 was elected leader, sending a full tablet report...
W20260808 16:59:39.562143 29801 tablet.cc:2369] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:59:39.656095 29606 tablet.cc:2369] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:59:39.713758 30008 tablet.cc:2369] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:59:39.789398 30044 raft_consensus.cc:482] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:59:39.789533 30044 raft_consensus.cc:504] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } }
I20260808 16:59:39.789886 30044 leader_election.cc:288] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721), bb8fbeac543346de8fd0593d3f7b59dd (127.8.14.65:33131)
I20260808 16:59:39.792903 29802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.790068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:35721 (local address 127.8.14.66:39845)
0808 16:59:39.790131 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:39.790144 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:39.790203 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:39.790400 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:39.790401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:39.790402 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:39.790481 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:39.790483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.791251 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.791253 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:39.792019 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:39.792022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.792595 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.792596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:39.792717 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.792721 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:59:39.792778 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 16:59:39.792825 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 16:59:39.792922 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.790196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:39845 (local address 127.8.14.67:35721)
0808 16:59:39.790267 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:39.790270 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:39.790279 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:59:39.790318 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:39.790319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:39.790330 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:39.790394 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:39.790552 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.791086 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.792119 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.792509 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.792594 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.792605 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:39.792807 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 16:59:39.792860 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 16:59:39.793093 30049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.790068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:33131 (local address 127.8.14.66:45259)
0808 16:59:39.790529 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:39.790539 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:39.790620 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:39.790801 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:39.790802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:39.790804 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:39.790880 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:39.790882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.791536 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.791537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:39.792283 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:39.792285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.792837 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.792838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:39.792905 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.792909 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:59:39.792963 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 16:59:39.793000 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":268,"thread_start_us":126,"threads_started":1}
I20260808 16:59:39.793110 30033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.790207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:45259 (local address 127.8.14.65:33131)
0808 16:59:39.790252 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:39.790254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:39.790611 (+ 357us) server_negotiation.cc:407] Connection header received
0808 16:59:39.790721 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:39.790722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:39.790738 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:39.790795 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:39.790971 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.791448 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.792376 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.792739 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.792817 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.792827 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:39.792981 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 16:59:39.793029 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 16:59:39.793358 29930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79343eed25034ea39b87b0403aa74c8f" candidate_uuid: "e877162b46d64529850b1dc1a0bd79a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a5955432af4f1ebd619d258ec5e829" is_pre_election: true
I20260808 16:59:39.793537 29930 raft_consensus.cc:2454] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e877162b46d64529850b1dc1a0bd79a2 in term 0.
I20260808 16:59:39.793529 29536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79343eed25034ea39b87b0403aa74c8f" candidate_uuid: "e877162b46d64529850b1dc1a0bd79a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" is_pre_election: true
I20260808 16:59:39.793680 29536 raft_consensus.cc:2454] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e877162b46d64529850b1dc1a0bd79a2 in term 0.
I20260808 16:59:39.793795 29639 leader_election.cc:302] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a5955432af4f1ebd619d258ec5e829, e877162b46d64529850b1dc1a0bd79a2; no voters:
I20260808 16:59:39.793927 30044 raft_consensus.cc:2781] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:59:39.793985 30044 raft_consensus.cc:482] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:59:39.794009 30044 raft_consensus.cc:3037] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:39.794711 30044 raft_consensus.cc:504] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } }
I20260808 16:59:39.794878 30044 leader_election.cc:288] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [CANDIDATE]: Term 1 election: Requested vote from peers 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721), bb8fbeac543346de8fd0593d3f7b59dd (127.8.14.65:33131)
I20260808 16:59:39.795053 29930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79343eed25034ea39b87b0403aa74c8f" candidate_uuid: "e877162b46d64529850b1dc1a0bd79a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a5955432af4f1ebd619d258ec5e829"
I20260808 16:59:39.795140 29930 raft_consensus.cc:3037] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:39.795975 29930 raft_consensus.cc:2454] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e877162b46d64529850b1dc1a0bd79a2 in term 1.
I20260808 16:59:39.796137 29536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79343eed25034ea39b87b0403aa74c8f" candidate_uuid: "e877162b46d64529850b1dc1a0bd79a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb8fbeac543346de8fd0593d3f7b59dd"
I20260808 16:59:39.796154 29639 leader_election.cc:302] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a5955432af4f1ebd619d258ec5e829, e877162b46d64529850b1dc1a0bd79a2; no voters:
I20260808 16:59:39.796221 29536 raft_consensus.cc:3037] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:39.796264 30044 raft_consensus.cc:2781] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:59:39.796602 30044 raft_consensus.cc:686] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [term 1 LEADER]: Becoming Leader. State: Replica: e877162b46d64529850b1dc1a0bd79a2, State: Running, Role: LEADER
I20260808 16:59:39.796720 30044 consensus_queue.cc:237] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } }
I20260808 16:59:39.797075 29536 raft_consensus.cc:2454] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e877162b46d64529850b1dc1a0bd79a2 in term 1.
I20260808 16:59:39.797669 29310 catalog_manager.cc:5495] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66). New cstate: current_term: 1 leader_uuid: "e877162b46d64529850b1dc1a0bd79a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } health_report { overall_health: HEALTHY } } }
I20260808 16:59:39.851524 30032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.849044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60364 (local address 127.8.14.66:46293)
0808 16:59:39.849109 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:39.849111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:39.849127 (+ 16us) server_negotiation.cc:407] Connection header received
0808 16:59:39.849167 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:39.849168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:39.849195 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:39.849287 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:59:39.849435 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.849914 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.850691 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:39.850916 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:39.850989 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:39.851153 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:59:39.851266 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:59:39.851360 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:39.851380 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 16:59:39.851401 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 16:59:39.853439 29536 raft_consensus.cc:1261] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Refusing update from remote peer e877162b46d64529850b1dc1a0bd79a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:59:39.853439 29930 raft_consensus.cc:1261] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829 [term 1 FOLLOWER]: Refusing update from remote peer e877162b46d64529850b1dc1a0bd79a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:59:39.853853 30052 consensus_queue.cc:1035] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:59:39.854010 30044 consensus_queue.cc:1035] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:59:39.858870 29309 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:59:39.859027 29309 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:59:39.859746 29309 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db: Sending DeleteTablet for 3 replicas of tablet 79343eed25034ea39b87b0403aa74c8f
I20260808 16:59:39.860092 29708 tablet_service.cc:1514] Processing DeleteTablet for tablet 79343eed25034ea39b87b0403aa74c8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:59:39 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:60354
I20260808 16:59:39.860117 29902 tablet_service.cc:1514] Processing DeleteTablet for tablet 79343eed25034ea39b87b0403aa74c8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:59:39 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:48670
I20260808 16:59:39.860116 29508 tablet_service.cc:1514] Processing DeleteTablet for tablet 79343eed25034ea39b87b0403aa74c8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:59:39 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:33454
I20260808 16:59:39.860349 30066 tablet_replica.cc:330] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2: stopping tablet replica
I20260808 16:59:39.860458 30066 raft_consensus.cc:2229] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:59:39.860463 30068 tablet_replica.cc:330] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: stopping tablet replica
I20260808 16:59:39.860491 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 29411
I20260808 16:59:39.868678 30066 raft_consensus.cc:2258] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:59:39.869374 30070 tablet_replica.cc:330] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829: stopping tablet replica
I20260808 16:59:39.869448 30070 raft_consensus.cc:2229] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:59:39.869520 30070 raft_consensus.cc:2258] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:59:39.870289 30070 ts_tablet_manager.cc:1905] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:59:39.872267 30070 ts_tablet_manager.cc:1918] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:59:39.872331 30070 log.cc:1196] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/79343eed25034ea39b87b0403aa74c8f
I20260808 16:59:39.872573 30070 ts_tablet_manager.cc:1939] T 79343eed25034ea39b87b0403aa74c8f P 62a5955432af4f1ebd619d258ec5e829: Deleting consensus metadata
I20260808 16:59:39.872996 29293 catalog_manager.cc:4841] TS 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721): tablet 79343eed25034ea39b87b0403aa74c8f (table test-table [id=75271de0384c4cb6aef45bbeca0cc0f3]) successfully deleted
I20260808 16:59:39.875686 30066 ts_tablet_manager.cc:1905] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:59:39.877576 30066 ts_tablet_manager.cc:1918] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:59:39.877655 30066 log.cc:1196] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/79343eed25034ea39b87b0403aa74c8f
I20260808 16:59:39.877902 30066 ts_tablet_manager.cc:1939] T 79343eed25034ea39b87b0403aa74c8f P e877162b46d64529850b1dc1a0bd79a2: Deleting consensus metadata
I20260808 16:59:39.878299 29288 catalog_manager.cc:4841] TS e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66:46293): tablet 79343eed25034ea39b87b0403aa74c8f (table test-table [id=75271de0384c4cb6aef45bbeca0cc0f3]) successfully deleted
W20260808 16:59:39.880789 29288 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.14.65:33131 (error 108)
I20260808 16:59:39.880995 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 29610
I20260808 16:59:39.881305 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:39.881162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:33131 (local address 127.0.0.1:33464)
0808 16:59:39.881216 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:39.881258 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.65:33131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 16:59:39.883075 29288 catalog_manager.cc:4584] TS bb8fbeac543346de8fd0593d3f7b59dd (127.8.14.65:33131): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 79343eed25034ea39b87b0403aa74c8f: Network error: Client connection negotiation failed: client connection to 127.8.14.65:33131: connect: Connection refused (error 111)
I20260808 16:59:39.892469 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 29806
I20260808 16:59:39.899501 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 29258
I20260808 16:59:39.911391 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:45175
--webserver_interface=127.8.14.126
--webserver_port=40643
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:45175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:59:39.994056 30072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:39.994285 30072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:39.994325 30072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:39.995486 30072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:59:39.995550 30072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:39.995575 30072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:39.995597 30072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:59:39.995625 30072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:59:39.997067 30072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:45175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:45175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=40643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:39.997335 30072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:39.997550 30072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:40.007019 30086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:40.007012 30089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:40.007272 30072 server_base.cc:1034] running on GCE node
W20260808 16:59:40.007012 30085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:40.007573 30072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:40.007896 30072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:40.009058 30072 hybrid_clock.cc:648] HybridClock initialized: now 1786208380009053 us; error 35 us; skew 500 ppm
Aug 08 16:59:40 dist-test-slave-1nl6 krb5kdc[29238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208380, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:40.011576 30072 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:59:40.012023 30072 webserver.cc:460] Webserver started at http://127.8.14.126:40643/ using document root <none> and password file <none>
I20260808 16:59:40.012267 30072 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:40.012347 30072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:40.013700 30072 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:40.014341 30101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:40.014511 30072 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:40.014616 30072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "87f8e814f3314147bd30d9eb7d9382db"
format_stamp: "Formatted at 2026-08-08 16:59:38 on dist-test-slave-1nl6"
server_key: "8d52ae1c84779e3f89c02285eca8bab1"
server_key_iv: "71eb4ae72d0dca6c10b73a767b13ad41"
server_key_version: "encryptionkey@0"
I20260808 16:59:40.015094 30072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:40.029412 30072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:40.031206 30072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:40.031371 30072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:40.035382 30072 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:45175
I20260808 16:59:40.035418 30179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:45175 every 8 connection(s)
I20260808 16:59:40.035831 30072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:59:40.036545 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 30072
I20260808 16:59:40.036840 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:33131
--local_ip_for_outbound_sockets=127.8.14.65
--tserver_master_addrs=127.8.14.126:45175
--webserver_port=37849
--webserver_interface=127.8.14.65
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:59:40.037688 30181 sys_catalog.cc:263] Verifying existing consensus state
I20260808 16:59:40.038434 30181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db: Bootstrap starting.
I20260808 16:59:40.047385 30181 log.cc:824] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:40.049310 30181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:59:40.049624 30181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db: Bootstrap complete.
I20260808 16:59:40.051964 30181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } }
I20260808 16:59:40.052222 30181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87f8e814f3314147bd30d9eb7d9382db, State: Initialized, Role: FOLLOWER
I20260808 16:59:40.052399 30181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } }
I20260808 16:59:40.052464 30181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:59:40.052517 30181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:59:40.052569 30181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 1 FOLLOWER]: Advancing to term 2
I20260808 16:59:40.053578 30181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } }
I20260808 16:59:40.053694 30181 leader_election.cc:302] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87f8e814f3314147bd30d9eb7d9382db; no voters:
I20260808 16:59:40.053901 30181 leader_election.cc:288] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 16:59:40.053941 30187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 2 FOLLOWER]: Leader election won for term 2
I20260808 16:59:40.054112 30187 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 2 LEADER]: Becoming Leader. State: Replica: 87f8e814f3314147bd30d9eb7d9382db, State: Running, Role: LEADER
I20260808 16:59:40.054174 30181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:59:40.054293 30187 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } }
I20260808 16:59:40.054527 30187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87f8e814f3314147bd30d9eb7d9382db. Latest consensus state: current_term: 2 leader_uuid: "87f8e814f3314147bd30d9eb7d9382db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } } }
I20260808 16:59:40.054534 30191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "87f8e814f3314147bd30d9eb7d9382db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } } }
I20260808 16:59:40.054605 30187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [sys.catalog]: This master's current role is: LEADER
I20260808 16:59:40.054642 30191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [sys.catalog]: This master's current role is: LEADER
I20260808 16:59:40.054808 30196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:59:40.055562 30196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:59:40.055720 30196 catalog_manager.cc:1259] Loaded cluster ID: 81bc815e5acd466597eec2d3e7dd71ad
I20260808 16:59:40.056033 30196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:59:40.056286 30196 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 16:59:40.060783 30196 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 16:59:40.063482 30196 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:59:40.063664 30196 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db: Loaded TSK: 0
I20260808 16:59:40.063867 30196 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 16:59:40.123896 30184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:40.124094 30184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:40.124118 30184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:40.125352 30184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:40.125394 30184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:40.125422 30184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:59:40.126870 30184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:33131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=37849
--tserver_master_addrs=127.8.14.126:45175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:40.127137 30184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:40.127393 30184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:40.133355 30220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:40.133474 30219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:40.133621 30222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:40.133690 30184 server_base.cc:1034] running on GCE node
I20260808 16:59:40.133991 30184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:40.134280 30184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:40.135465 30184 hybrid_clock.cc:648] HybridClock initialized: now 1786208380135459 us; error 41 us; skew 500 ppm
Aug 08 16:59:40 dist-test-slave-1nl6 krb5kdc[29238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208380, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:40.138331 30184 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:59:40.138976 30184 webserver.cc:460] Webserver started at http://127.8.14.65:37849/ using document root <none> and password file <none>
I20260808 16:59:40.139279 30184 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:40.139356 30184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:40.141301 30184 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:59:40.142232 30232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:40.142423 30184 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:40.142508 30184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "bb8fbeac543346de8fd0593d3f7b59dd"
format_stamp: "Formatted at 2026-08-08 16:59:39 on dist-test-slave-1nl6"
server_key: "9905d26121efbb62cfbced5d405290b7"
server_key_iv: "802972def4b9dc1cee0f9ac72f71b0aa"
server_key_version: "encryptionkey@0"
I20260808 16:59:40.142930 30184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:40.162624 30184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:40.165097 30184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:40.165249 30184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:40.165503 30184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:40.165983 30243 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 16:59:40.167500 30184 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 16:59:40.167569 30184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 16:59:40.167603 30184 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 16:59:40.168475 30184 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 16:59:40.168526 30184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 16:59:40.168586 30243 tablet_bootstrap.cc:492] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: Bootstrap starting.
I20260808 16:59:40.175920 30184 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:33131
I20260808 16:59:40.176165 30399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:33131 every 8 connection(s)
I20260808 16:59:40.176373 30184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:59:40.179250 30243 log.cc:824] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:40.181663 30243 tablet_bootstrap.cc:492] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:59:40.181968 30243 tablet_bootstrap.cc:492] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: Bootstrap complete.
I20260808 16:59:40.183017 30243 ts_tablet_manager.cc:1397] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.006s
I20260808 16:59:40.183239 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 30184
I20260808 16:59:40.183552 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:46293
--local_ip_for_outbound_sockets=127.8.14.66
--tserver_master_addrs=127.8.14.126:45175
--webserver_port=45707
--webserver_interface=127.8.14.66
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:59:40.184698 30243 raft_consensus.cc:348] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } }
I20260808 16:59:40.185108 30243 raft_consensus.cc:729] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb8fbeac543346de8fd0593d3f7b59dd, State: Initialized, Role: FOLLOWER
I20260808 16:59:40.185297 30243 consensus_queue.cc:260] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } }
I20260808 16:59:40.185590 30243 ts_tablet_manager.cc:1428] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
Aug 08 16:59:40 dist-test-slave-1nl6 krb5kdc[29238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208380, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:40.191339 30403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.177256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:43289)
0808 16:59:40.178872 (+ 1616us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:40.178950 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:40.179592 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:40.180088 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:40.180092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:40.180438 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:40.180680 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:40.180685 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.182342 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.182343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:40.184035 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:40.184038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.184896 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.184897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:40.185035 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.185774 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:40.185786 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:40.187708 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:40.189724 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:40.189731 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:40.189733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:40.189954 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:40.190223 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:40.190223 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:40.190224 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:40.190335 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:40.190695 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:40.190702 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:40.190836 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 16:59:40.190965 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 16:59:40.191143 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1570,"thread_start_us":105,"threads_started":1}
I20260808 16:59:40.191339 30404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.177366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:43289 (local address 127.8.14.126:45175)
0808 16:59:40.179049 (+ 1683us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:40.179058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:40.179100 (+ 42us) server_negotiation.cc:407] Connection header received
0808 16:59:40.179824 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:40.179831 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:40.179889 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:40.180034 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:40.180823 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.182225 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.184142 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.184678 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.184759 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.187823 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:40.187827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:40.187827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:40.187836 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:40.189619 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:40.190041 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:40.190044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:40.190044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:40.190136 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:40.190409 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:40.190410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:40.190410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:40.190616 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:40.190701 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:40.191040 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 16:59:40.191164 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1618,"thread_start_us":97,"threads_started":1}
I20260808 16:59:40.192077 30400 heartbeater.cc:347] Connected to a master server at 127.8.14.126:45175
I20260808 16:59:40.192190 30400 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:40.192368 30400 heartbeater.cc:511] Master 127.8.14.126:45175 requested a full tablet report, sending...
I20260808 16:59:40.193030 30125 ts_manager.cc:194] Registered new tserver with Master: bb8fbeac543346de8fd0593d3f7b59dd (127.8.14.65:33131)
I20260808 16:59:40.194046 30125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:43289
I20260808 16:59:40.195140 30400 heartbeater.cc:503] Master 127.8.14.126:45175 was elected leader, sending a full tablet report...
I20260808 16:59:40.197966 30409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.194330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33474 (local address 127.8.14.65:33131)
0808 16:59:40.194480 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:40.194484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:40.194819 (+ 335us) server_negotiation.cc:407] Connection header received
0808 16:59:40.194854 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:40.194856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:40.194990 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:40.195062 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:40.195379 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.196154 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.197103 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.197543 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.197624 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.197675 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:40.197862 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 16:59:40.197899 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260808 16:59:40.197978 30408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.193588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:33131 (local address 127.0.0.1:33474)
0808 16:59:40.194628 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:40.194664 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:40.194736 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:40.195120 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:40.195123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:40.195125 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:40.195185 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:40.195187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.196240 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.196241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:40.197011 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:40.197013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.197638 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.197640 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:40.197723 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.197735 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 16:59:40.197844 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 16:59:40.197899 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1013,"thread_start_us":88,"threads_started":1}
I20260808 16:59:40.198601 30301 tablet_service.cc:1514] Processing DeleteTablet for tablet 79343eed25034ea39b87b0403aa74c8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:59:39 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:33474
I20260808 16:59:40.199091 30410 tablet_replica.cc:330] stopping tablet replica
I20260808 16:59:40.199209 30410 raft_consensus.cc:2229] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:59:40.199275 30410 raft_consensus.cc:2258] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:59:40.199613 30410 ts_tablet_manager.cc:1905] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:59:40.201388 30410 ts_tablet_manager.cc:1918] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:59:40.201472 30410 log.cc:1196] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal/wals/79343eed25034ea39b87b0403aa74c8f
I20260808 16:59:40.201696 30410 ts_tablet_manager.cc:1939] T 79343eed25034ea39b87b0403aa74c8f P bb8fbeac543346de8fd0593d3f7b59dd: Deleting consensus metadata
I20260808 16:59:40.202152 30102 catalog_manager.cc:4841] TS bb8fbeac543346de8fd0593d3f7b59dd (127.8.14.65:33131): tablet 79343eed25034ea39b87b0403aa74c8f (table test-table [id=75271de0384c4cb6aef45bbeca0cc0f3]) successfully deleted
W20260808 16:59:40.270080 30406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:40.270277 30406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:40.270305 30406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:40.271457 30406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:40.271502 30406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:40.271528 30406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:59:40.272997 30406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:46293
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=45707
--tserver_master_addrs=127.8.14.126:45175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:40.273224 30406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:40.273461 30406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:40.278429 30418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:40.278462 30423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:40.278429 30419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:40.278541 30406 server_base.cc:1034] running on GCE node
I20260808 16:59:40.278960 30406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:40.279219 30406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:40.280382 30406 hybrid_clock.cc:648] HybridClock initialized: now 1786208380280377 us; error 40 us; skew 500 ppm
Aug 08 16:59:40 dist-test-slave-1nl6 krb5kdc[29238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208380, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:40.282976 30406 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:59:40.283356 30406 webserver.cc:460] Webserver started at http://127.8.14.66:45707/ using document root <none> and password file <none>
I20260808 16:59:40.283550 30406 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:40.283600 30406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:40.284821 30406 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:40.285359 30430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:40.285547 30406 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:40.285638 30406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "e877162b46d64529850b1dc1a0bd79a2"
format_stamp: "Formatted at 2026-08-08 16:59:39 on dist-test-slave-1nl6"
server_key: "d266bbf42be5e7216cd2e10cae12e5d4"
server_key_iv: "823efe4ffdacad2f9314dd495b6f6b7d"
server_key_version: "encryptionkey@0"
I20260808 16:59:40.285949 30406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:40.303400 30406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:40.305274 30406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:40.305464 30406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:40.305694 30406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:40.306005 30406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:59:40.306039 30406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:40.306079 30406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:59:40.306106 30406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:40.311892 30406 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:46293
I20260808 16:59:40.311980 30592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:46293 every 8 connection(s)
I20260808 16:59:40.312317 30406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
Aug 08 16:59:40 dist-test-slave-1nl6 krb5kdc[29238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208380, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:40.321278 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 30406
I20260808 16:59:40.321616 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:35721
--local_ip_for_outbound_sockets=127.8.14.67
--tserver_master_addrs=127.8.14.126:45175
--webserver_port=39041
--webserver_interface=127.8.14.67
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:59:40.321693 30404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.313121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:42887 (local address 127.8.14.126:45175)
0808 16:59:40.313184 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:40.313186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:40.313410 (+ 224us) server_negotiation.cc:407] Connection header received
0808 16:59:40.313957 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:40.313958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:40.313975 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:40.314026 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:40.315022 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.315533 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.316527 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.316730 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.316813 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.319071 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:40.319072 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:40.319072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:40.319079 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:40.320321 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:40.320701 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:40.320702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:40.320703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:40.320755 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:40.320979 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:40.320980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:40.320980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:40.321081 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:40.321132 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:40.321532 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 16:59:40.321611 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 16:59:40.322042 30596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.313108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:42887)
0808 16:59:40.313289 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:40.313327 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:40.313842 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:40.314096 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:40.314100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:40.314685 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:40.314889 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:40.314938 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.315639 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.315640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:40.316439 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:40.316441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.316828 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.316829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:40.316956 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.317491 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:40.317499 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:40.318979 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:40.320401 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:40.320407 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:40.320409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:40.320638 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:40.320811 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:40.320812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:40.320812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:40.320919 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:40.321176 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:40.321185 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:40.321382 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 16:59:40.321517 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 16:59:40.321701 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
I20260808 16:59:40.322585 30593 heartbeater.cc:347] Connected to a master server at 127.8.14.126:45175
I20260808 16:59:40.322692 30593 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:40.322918 30593 heartbeater.cc:511] Master 127.8.14.126:45175 requested a full tablet report, sending...
I20260808 16:59:40.323325 30125 ts_manager.cc:194] Registered new tserver with Master: e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66:46293)
I20260808 16:59:40.323707 30125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:42887
W20260808 16:59:40.404991 30597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:40.405215 30597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:40.405253 30597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:40.406373 30597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:40.406433 30597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:40.406474 30597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:59:40.407889 30597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:35721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=39041
--tserver_master_addrs=127.8.14.126:45175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:40.408175 30597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:40.408398 30597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:40.413269 30605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:40.413275 30608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:40.413285 30606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:40.413450 30597 server_base.cc:1034] running on GCE node
I20260808 16:59:40.413690 30597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:40.413887 30597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:40.415015 30597 hybrid_clock.cc:648] HybridClock initialized: now 1786208380415005 us; error 27 us; skew 500 ppm
Aug 08 16:59:40 dist-test-slave-1nl6 krb5kdc[29238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208380, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:40.417470 30597 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:59:40.417912 30597 webserver.cc:460] Webserver started at http://127.8.14.67:39041/ using document root <none> and password file <none>
I20260808 16:59:40.418179 30597 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:40.418280 30597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:40.419641 30597 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260808 16:59:40.420198 30618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:40.420362 30597 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:59:40.420456 30597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "62a5955432af4f1ebd619d258ec5e829"
format_stamp: "Formatted at 2026-08-08 16:59:39 on dist-test-slave-1nl6"
server_key: "0b155a473ddb646aee7bfcfef1d57eae"
server_key_iv: "483beff439602a9fd461293f03f7ba3e"
server_key_version: "encryptionkey@0"
I20260808 16:59:40.420766 30597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:40.432847 30597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:40.434489 30597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:40.434636 30597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:40.434849 30597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:40.435158 30597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:59:40.435215 30597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:40.435254 30597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:59:40.435282 30597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:40.441043 30597 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:35721
I20260808 16:59:40.441087 30777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:35721 every 8 connection(s)
I20260808 16:59:40.441404 30597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
Aug 08 16:59:40 dist-test-slave-1nl6 krb5kdc[29238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208380, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:40.447990 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 30597
I20260808 16:59:40.450250 30404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.442083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:50051 (local address 127.8.14.126:45175)
0808 16:59:40.442146 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:40.442148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:40.442340 (+ 192us) server_negotiation.cc:407] Connection header received
0808 16:59:40.442759 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:40.442760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:40.442771 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:40.442830 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:40.443726 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.444218 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.445121 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.445294 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.445345 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.447458 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:40.447460 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:40.447460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:40.447467 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:40.448895 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:40.449261 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:40.449262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:40.449262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:40.449332 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:40.449609 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:40.449610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:40.449610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:40.449708 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:40.449753 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:40.450073 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 16:59:40.450157 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 16:59:40.450376 30781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.442080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:50051)
0808 16:59:40.442227 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:40.442259 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:40.442680 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:40.442880 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:40.442883 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:40.443470 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:40.443641 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:40.443645 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.444290 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.444291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:40.445052 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:40.445054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.445364 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.445365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:40.445457 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.445928 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:40.445936 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:40.447352 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:40.448964 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:40.448970 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:40.448971 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:40.449175 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:40.449393 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:40.449393 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:40.449394 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:40.449535 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:40.449767 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:40.449774 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:40.449936 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 16:59:40.450036 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 16:59:40.450190 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260808 16:59:40.450853 30778 heartbeater.cc:347] Connected to a master server at 127.8.14.126:45175
I20260808 16:59:40.450976 30778 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:40.451153 30778 heartbeater.cc:511] Master 127.8.14.126:45175 requested a full tablet report, sending...
I20260808 16:59:40.451575 30125 ts_manager.cc:194] Registered new tserver with Master: 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721)
I20260808 16:59:40.451963 30125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:50051
I20260808 16:59:40.455147 30784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.448374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37708 (local address 127.8.14.126:45175)
0808 16:59:40.448495 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:40.448497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:40.449078 (+ 581us) server_negotiation.cc:407] Connection header received
0808 16:59:40.449169 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:40.449169 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:40.449180 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:40.449237 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:40.449946 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.450644 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.451669 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.451894 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.451948 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.453095 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:40.453096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:40.453096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:40.453102 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:40.454278 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:40.454593 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:40.454594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:40.454594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:40.454637 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:40.454743 (+ 106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:40.454744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:40.454744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:40.454829 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:40.454990 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:40.455029 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 16:59:40.455089 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":68,"threads_started":1}
I20260808 16:59:40.455726 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:59:40.458379 30784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.456069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37718 (local address 127.8.14.126:45175)
0808 16:59:40.456123 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:40.456124 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:40.456212 (+ 88us) server_negotiation.cc:407] Connection header received
0808 16:59:40.456286 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:40.456287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:40.456321 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:40.456396 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:59:40.456497 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.456928 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.457644 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.457851 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.457925 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.458099 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:59:40.458184 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:59:40.458268 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:40.458290 (+ 22us) server_negotiation.cc:299] Negotiation successful
0808 16:59:40.458310 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 16:59:40.458665 30125 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:37718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:59:40.460264 30125 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:59:40.462332 30301 tablet_service.cc:1467] Processing CreateTablet for tablet 4f2d4cb3b3564d3195557116341d25e7 (DEFAULT_TABLE table=test-table [id=346f21da70514e688f788c3f1e544484]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:59:40.462534 30301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f2d4cb3b3564d3195557116341d25e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:40.464149 30243 tablet_bootstrap.cc:492] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd: Bootstrap starting.
I20260808 16:59:40.464483 30243 tablet_bootstrap.cc:654] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:40.465087 30243 tablet_bootstrap.cc:492] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd: No bootstrap required, opened a new log
I20260808 16:59:40.465152 30243 ts_tablet_manager.cc:1397] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 16:59:40.465261 30243 raft_consensus.cc:348] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } }
I20260808 16:59:40.465337 30243 raft_consensus.cc:374] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:40.465385 30243 raft_consensus.cc:729] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb8fbeac543346de8fd0593d3f7b59dd, State: Initialized, Role: FOLLOWER
I20260808 16:59:40.465442 30243 consensus_queue.cc:260] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } }
I20260808 16:59:40.465549 30243 ts_tablet_manager.cc:1428] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:40.465687 30408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.461876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:35721 (local address 127.0.0.1:48676)
0808 16:59:40.461908 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:40.461919 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:40.461959 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:40.462449 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:40.462451 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:40.462452 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:40.462488 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:40.462489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.463587 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.463588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:40.464392 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:40.464395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.465468 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.465469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:40.465529 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.465533 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:59:40.465567 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 16:59:40.465610 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":23}
I20260808 16:59:40.465997 30793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.462139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48676 (local address 127.8.14.67:35721)
0808 16:59:40.462300 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:40.462304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:40.462317 (+ 13us) server_negotiation.cc:407] Connection header received
0808 16:59:40.462344 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:40.462345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:40.462356 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:40.462436 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:40.462563 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.463488 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.464509 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.465360 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.465798 (+ 438us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.465864 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:40.465890 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 16:59:40.465925 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260808 16:59:40.466122 30790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.461942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:46293 (local address 127.0.0.1:60376)
0808 16:59:40.462067 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:40.462077 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:40.462163 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:40.462529 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:40.462530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:40.462532 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:40.462565 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:40.462566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.463763 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.463764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:40.464927 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:40.464931 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.465884 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.465886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:40.465948 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.465952 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:59:40.465981 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 16:59:40.466022 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260808 16:59:40.466464 30791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.462042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60376 (local address 127.8.14.66:46293)
0808 16:59:40.462338 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:40.462342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:40.462354 (+ 12us) server_negotiation.cc:407] Connection header received
0808 16:59:40.462393 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:40.462395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:40.462411 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:40.462530 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:40.462615 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.463636 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.465036 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.465770 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.466287 (+ 517us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.466357 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:40.466378 (+ 21us) server_negotiation.cc:299] Negotiation successful
0808 16:59:40.466407 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":171,"threads_started":1}
I20260808 16:59:40.466532 30685 tablet_service.cc:1467] Processing CreateTablet for tablet 4f2d4cb3b3564d3195557116341d25e7 (DEFAULT_TABLE table=test-table [id=346f21da70514e688f788c3f1e544484]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:59:40.466887 30685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f2d4cb3b3564d3195557116341d25e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:40.467159 30500 tablet_service.cc:1467] Processing CreateTablet for tablet 4f2d4cb3b3564d3195557116341d25e7 (DEFAULT_TABLE table=test-table [id=346f21da70514e688f788c3f1e544484]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:59:40.467450 30500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f2d4cb3b3564d3195557116341d25e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:40.469583 30796 tablet_bootstrap.cc:492] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829: Bootstrap starting.
I20260808 16:59:40.470283 30796 tablet_bootstrap.cc:654] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:40.470295 30798 tablet_bootstrap.cc:492] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2: Bootstrap starting.
I20260808 16:59:40.470777 30796 log.cc:824] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:40.470997 30798 tablet_bootstrap.cc:654] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:40.471525 30798 log.cc:824] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:40.471653 30796 tablet_bootstrap.cc:492] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829: No bootstrap required, opened a new log
I20260808 16:59:40.471735 30796 ts_tablet_manager.cc:1397] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:59:40.472211 30798 tablet_bootstrap.cc:492] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2: No bootstrap required, opened a new log
I20260808 16:59:40.472265 30798 ts_tablet_manager.cc:1397] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 16:59:40.473084 30796 raft_consensus.cc:348] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } }
I20260808 16:59:40.473235 30796 raft_consensus.cc:374] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:40.473281 30796 raft_consensus.cc:729] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62a5955432af4f1ebd619d258ec5e829, State: Initialized, Role: FOLLOWER
I20260808 16:59:40.473423 30796 consensus_queue.cc:260] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } }
I20260808 16:59:40.473683 30796 ts_tablet_manager.cc:1428] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:59:40.473744 30778 heartbeater.cc:503] Master 127.8.14.126:45175 was elected leader, sending a full tablet report...
I20260808 16:59:40.473659 30798 raft_consensus.cc:348] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } }
I20260808 16:59:40.473834 30798 raft_consensus.cc:374] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:40.473879 30798 raft_consensus.cc:729] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e877162b46d64529850b1dc1a0bd79a2, State: Initialized, Role: FOLLOWER
I20260808 16:59:40.473992 30798 consensus_queue.cc:260] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } }
I20260808 16:59:40.474268 30798 ts_tablet_manager.cc:1428] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 16:59:40.474491 30593 heartbeater.cc:503] Master 127.8.14.126:45175 was elected leader, sending a full tablet report...
W20260808 16:59:40.562871 30594 tablet.cc:2369] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:59:40.600826 30805 raft_consensus.cc:482] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:59:40.600947 30805 raft_consensus.cc:504] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } }
I20260808 16:59:40.601382 30805 leader_election.cc:288] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721), bb8fbeac543346de8fd0593d3f7b59dd (127.8.14.65:33131)
I20260808 16:59:40.604355 30596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.601544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:35721 (local address 127.8.14.66:52851)
0808 16:59:40.601617 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:40.601630 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:40.601675 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:40.601851 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:40.601852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:40.601853 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:40.601927 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:40.601929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.602555 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.602556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:40.603433 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:40.603436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.604148 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.604149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:40.604209 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.604212 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:59:40.604252 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 16:59:40.604282 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":27}
I20260808 16:59:40.604415 30793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.601660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:52851 (local address 127.8.14.67:35721)
0808 16:59:40.601714 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:40.601716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:40.601724 (+ 8us) server_negotiation.cc:407] Connection header received
0808 16:59:40.601752 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:40.601753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:40.601765 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:40.601831 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:40.601998 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.602472 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.603530 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.604060 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.604131 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.604141 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:40.604287 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 16:59:40.604340 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 16:59:40.604671 30409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.601652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:56119 (local address 127.8.14.65:33131)
0808 16:59:40.601730 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:40.601733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:40.601851 (+ 118us) server_negotiation.cc:407] Connection header received
0808 16:59:40.601936 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:40.601937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:40.601949 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:40.602020 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:40.602172 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.602634 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.603713 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.604300 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.604370 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.604379 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:40.604548 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 16:59:40.604595 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 16:59:40.604871 30806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.601578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:33131 (local address 127.8.14.66:56119)
0808 16:59:40.601778 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:40.601785 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:40.601867 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:40.602024 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:40.602025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:40.602026 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:40.602098 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:40.602099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.602718 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.602719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:40.603610 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:40.603613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.604384 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.604384 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:40.604454 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.604459 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:59:40.604759 (+ 300us) client_negotiation.cc:240] Negotiation successful
0808 16:59:40.604802 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":122,"threads_started":1}
I20260808 16:59:40.604944 30713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f2d4cb3b3564d3195557116341d25e7" candidate_uuid: "e877162b46d64529850b1dc1a0bd79a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a5955432af4f1ebd619d258ec5e829" is_pre_election: true
I20260808 16:59:40.605104 30713 raft_consensus.cc:2454] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e877162b46d64529850b1dc1a0bd79a2 in term 0.
I20260808 16:59:40.605170 30331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f2d4cb3b3564d3195557116341d25e7" candidate_uuid: "e877162b46d64529850b1dc1a0bd79a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" is_pre_election: true
I20260808 16:59:40.605333 30331 raft_consensus.cc:2454] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e877162b46d64529850b1dc1a0bd79a2 in term 0.
I20260808 16:59:40.605362 30436 leader_election.cc:302] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a5955432af4f1ebd619d258ec5e829, e877162b46d64529850b1dc1a0bd79a2; no voters:
I20260808 16:59:40.605468 30805 raft_consensus.cc:2781] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:59:40.605522 30805 raft_consensus.cc:482] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:59:40.605543 30805 raft_consensus.cc:3037] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:40.606483 30805 raft_consensus.cc:504] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } }
I20260808 16:59:40.606637 30805 leader_election.cc:288] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [CANDIDATE]: Term 1 election: Requested vote from peers 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721), bb8fbeac543346de8fd0593d3f7b59dd (127.8.14.65:33131)
I20260808 16:59:40.606823 30331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f2d4cb3b3564d3195557116341d25e7" candidate_uuid: "e877162b46d64529850b1dc1a0bd79a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb8fbeac543346de8fd0593d3f7b59dd"
I20260808 16:59:40.606842 30713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f2d4cb3b3564d3195557116341d25e7" candidate_uuid: "e877162b46d64529850b1dc1a0bd79a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62a5955432af4f1ebd619d258ec5e829"
I20260808 16:59:40.606942 30713 raft_consensus.cc:3037] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:40.606949 30331 raft_consensus.cc:3037] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:40.607604 30713 raft_consensus.cc:2454] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e877162b46d64529850b1dc1a0bd79a2 in term 1.
I20260808 16:59:40.607769 30436 leader_election.cc:302] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62a5955432af4f1ebd619d258ec5e829, e877162b46d64529850b1dc1a0bd79a2; no voters:
I20260808 16:59:40.607873 30805 raft_consensus.cc:2781] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:59:40.608022 30805 raft_consensus.cc:686] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [term 1 LEADER]: Becoming Leader. State: Replica: e877162b46d64529850b1dc1a0bd79a2, State: Running, Role: LEADER
I20260808 16:59:40.608124 30805 consensus_queue.cc:237] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } }
I20260808 16:59:40.608220 30331 raft_consensus.cc:2454] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e877162b46d64529850b1dc1a0bd79a2 in term 1.
I20260808 16:59:40.608959 30124 catalog_manager.cc:5495] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66). New cstate: current_term: 1 leader_uuid: "e877162b46d64529850b1dc1a0bd79a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } health_report { overall_health: UNKNOWN } } }
I20260808 16:59:40.646955 30791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.644615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60380 (local address 127.8.14.66:46293)
0808 16:59:40.644670 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:40.644672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:40.644683 (+ 11us) server_negotiation.cc:407] Connection header received
0808 16:59:40.644738 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:40.644738 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:40.644757 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:40.644845 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:59:40.644967 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.645418 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.646109 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:40.646302 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:40.646371 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:40.646569 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:59:40.646666 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:59:40.646759 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:40.646778 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 16:59:40.646808 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:59:40.648689 30713 raft_consensus.cc:1261] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829 [term 1 FOLLOWER]: Refusing update from remote peer e877162b46d64529850b1dc1a0bd79a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:59:40.648730 30331 raft_consensus.cc:1261] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Refusing update from remote peer e877162b46d64529850b1dc1a0bd79a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:59:40.649080 30805 consensus_queue.cc:1035] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:59:40.649245 30807 consensus_queue.cc:1035] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:59:40.651221 30813 mvcc.cc:205] Tried to move back new op lower bound from 7316309527134625792 to 7316309526971650048. Current Snapshot: MvccSnapshot[applied={T|T < 7316309527134625792}]
I20260808 16:59:40.652571 30817 mvcc.cc:205] Tried to move back new op lower bound from 7316309527134625792 to 7316309526971650048. Current Snapshot: MvccSnapshot[applied={T|T < 7316309527134625792}]
I20260808 16:59:40.653479 30124 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:37718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:59:40.653559 30124 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:37718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:59:40.654260 30124 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db: Sending DeleteTablet for 3 replicas of tablet 4f2d4cb3b3564d3195557116341d25e7
I20260808 16:59:40.654690 30500 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f2d4cb3b3564d3195557116341d25e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:59:40 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:60376
I20260808 16:59:40.654695 30685 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f2d4cb3b3564d3195557116341d25e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:59:40 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:48676
I20260808 16:59:40.655004 30301 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f2d4cb3b3564d3195557116341d25e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:59:40 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:33474
I20260808 16:59:40.655097 30828 tablet_replica.cc:330] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829: stopping tablet replica
I20260808 16:59:40.655087 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 30184
I20260808 16:59:40.667562 30828 raft_consensus.cc:2229] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:59:40.667690 30828 raft_consensus.cc:2258] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:59:40.667861 30829 tablet_replica.cc:330] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2: stopping tablet replica
I20260808 16:59:40.667953 30829 raft_consensus.cc:2229] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:59:40.668036 30829 raft_consensus.cc:2258] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:59:40.668747 30829 ts_tablet_manager.cc:1905] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:59:40.668747 30828 ts_tablet_manager.cc:1905] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:59:40.670449 30829 ts_tablet_manager.cc:1918] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:59:40.670449 30828 ts_tablet_manager.cc:1918] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:59:40.670533 30828 log.cc:1196] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/4f2d4cb3b3564d3195557116341d25e7
I20260808 16:59:40.670533 30829 log.cc:1196] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/4f2d4cb3b3564d3195557116341d25e7
I20260808 16:59:40.670855 30829 ts_tablet_manager.cc:1939] T 4f2d4cb3b3564d3195557116341d25e7 P e877162b46d64529850b1dc1a0bd79a2: Deleting consensus metadata
I20260808 16:59:40.670878 30828 ts_tablet_manager.cc:1939] T 4f2d4cb3b3564d3195557116341d25e7 P 62a5955432af4f1ebd619d258ec5e829: Deleting consensus metadata
I20260808 16:59:40.671317 30102 catalog_manager.cc:4841] TS e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66:46293): tablet 4f2d4cb3b3564d3195557116341d25e7 (table test-table [id=346f21da70514e688f788c3f1e544484]) successfully deleted
I20260808 16:59:40.671317 30105 catalog_manager.cc:4841] TS 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721): tablet 4f2d4cb3b3564d3195557116341d25e7 (table test-table [id=346f21da70514e688f788c3f1e544484]) successfully deleted
W20260808 16:59:40.674134 30102 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.14.65:33131 (error 108)
I20260808 16:59:40.674604 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 30406
I20260808 16:59:40.674652 30790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:40.674511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:33131 (local address 127.0.0.1:33484)
0808 16:59:40.674564 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:40.674615 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.65:33131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 16:59:40.680768 30102 catalog_manager.cc:4584] TS bb8fbeac543346de8fd0593d3f7b59dd (127.8.14.65:33131): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f2d4cb3b3564d3195557116341d25e7: Network error: Client connection negotiation failed: client connection to 127.8.14.65:33131: connect: Connection refused (error 111)
I20260808 16:59:40.685761 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 30597
I20260808 16:59:40.694118 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 30072
I20260808 16:59:40.706459 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:45175
--webserver_interface=127.8.14.126
--webserver_port=40643
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:45175
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:59:40.786656 30833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:40.786886 30833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:40.786942 30833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:40.788074 30833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:59:40.788137 30833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:40.788161 30833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:40.788192 30833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:59:40.788231 30833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:59:40.789752 30833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:45175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:45175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=40643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:40.790012 30833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:40.790227 30833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:40.795032 30842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:40.795084 30833 server_base.cc:1034] running on GCE node
W20260808 16:59:40.795024 30841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:40.795007 30844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:40.795392 30833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:40.795663 30833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:40.796823 30833 hybrid_clock.cc:648] HybridClock initialized: now 1786208380796818 us; error 32 us; skew 500 ppm
Aug 08 16:59:40 dist-test-slave-1nl6 krb5kdc[29238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208380, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:40.799170 30833 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:59:40.799612 30833 webserver.cc:460] Webserver started at http://127.8.14.126:40643/ using document root <none> and password file <none>
I20260808 16:59:40.799829 30833 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:40.799891 30833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:40.801126 30833 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:40.801867 30855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:40.802052 30833 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:40.802152 30833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "87f8e814f3314147bd30d9eb7d9382db"
format_stamp: "Formatted at 2026-08-08 16:59:38 on dist-test-slave-1nl6"
server_key: "8d52ae1c84779e3f89c02285eca8bab1"
server_key_iv: "71eb4ae72d0dca6c10b73a767b13ad41"
server_key_version: "encryptionkey@0"
I20260808 16:59:40.802457 30833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:40.812155 30833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:40.813846 30833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:40.813998 30833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:40.818068 30833 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:45175
I20260808 16:59:40.818097 30925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:45175 every 8 connection(s)
I20260808 16:59:40.818470 30833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:59:40.819991 30927 sys_catalog.cc:263] Verifying existing consensus state
I20260808 16:59:40.820670 30927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db: Bootstrap starting.
I20260808 16:59:40.821224 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 30833
I20260808 16:59:40.821523 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:33131
--local_ip_for_outbound_sockets=127.8.14.65
--tserver_master_addrs=127.8.14.126:45175
--webserver_port=37849
--webserver_interface=127.8.14.65
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:59:40.829524 30927 log.cc:824] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:40.831693 30927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:59:40.831984 30927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db: Bootstrap complete.
I20260808 16:59:40.834187 30927 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } }
I20260808 16:59:40.834430 30927 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87f8e814f3314147bd30d9eb7d9382db, State: Initialized, Role: FOLLOWER
I20260808 16:59:40.834605 30927 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } }
I20260808 16:59:40.834682 30927 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:59:40.834736 30927 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:59:40.834784 30927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 2 FOLLOWER]: Advancing to term 3
I20260808 16:59:40.835815 30927 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } }
I20260808 16:59:40.835934 30927 leader_election.cc:302] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87f8e814f3314147bd30d9eb7d9382db; no voters:
I20260808 16:59:40.836123 30927 leader_election.cc:288] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 16:59:40.836163 30931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 3 FOLLOWER]: Leader election won for term 3
I20260808 16:59:40.836282 30931 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [term 3 LEADER]: Becoming Leader. State: Replica: 87f8e814f3314147bd30d9eb7d9382db, State: Running, Role: LEADER
I20260808 16:59:40.836411 30927 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:59:40.836386 30931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } }
I20260808 16:59:40.836674 30931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87f8e814f3314147bd30d9eb7d9382db. Latest consensus state: current_term: 3 leader_uuid: "87f8e814f3314147bd30d9eb7d9382db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } } }
I20260808 16:59:40.836757 30931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [sys.catalog]: This master's current role is: LEADER
I20260808 16:59:40.836941 30932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "87f8e814f3314147bd30d9eb7d9382db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87f8e814f3314147bd30d9eb7d9382db" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45175 } } }
I20260808 16:59:40.837033 30932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87f8e814f3314147bd30d9eb7d9382db [sys.catalog]: This master's current role is: LEADER
I20260808 16:59:40.837383 30936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:59:40.838277 30936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:59:40.838388 30936 catalog_manager.cc:1259] Loaded cluster ID: 81bc815e5acd466597eec2d3e7dd71ad
I20260808 16:59:40.838423 30936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 16:59:40.838732 30936 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f85e9edf9ad google::LogMessage::Fail() at ??:0
@ 0x7f85e9ee288a google::LogMessage::SendToLog() at ??:0
@ 0x7f85e9edf4de google::LogMessage::Flush() at ??:0
@ 0x7f85e9ee0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f85f1e000f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f85f1e098b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f85ebdad0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f85ebda2954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f85ecc066db start_thread at ??:0
@ 0x7f85e89dd61f clone at ??:0
W20260808 16:59:40.942842 30929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:40.943140 30929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:40.943177 30929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:40.944887 30929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:40.944948 30929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:40.944985 30929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:59:40.947126 30929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:33131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=37849
--tserver_master_addrs=127.8.14.126:45175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:40.947395 30929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:40.947654 30929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:40.952941 30980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:40.952960 30979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:40.953166 30929 server_base.cc:1034] running on GCE node
W20260808 16:59:40.952958 30984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:40.953390 30929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:40.953647 30929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:40.954807 30929 hybrid_clock.cc:648] HybridClock initialized: now 1786208380954786 us; error 39 us; skew 500 ppm
Aug 08 16:59:40 dist-test-slave-1nl6 krb5kdc[29238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208380, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:40.957667 30929 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:59:40.958177 30929 webserver.cc:460] Webserver started at http://127.8.14.65:37849/ using document root <none> and password file <none>
I20260808 16:59:40.958485 30929 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:40.958560 30929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:40.960525 30929 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:40.961665 30995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:40.961874 30929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:40.961966 30929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "bb8fbeac543346de8fd0593d3f7b59dd"
format_stamp: "Formatted at 2026-08-08 16:59:39 on dist-test-slave-1nl6"
server_key: "9905d26121efbb62cfbced5d405290b7"
server_key_iv: "802972def4b9dc1cee0f9ac72f71b0aa"
server_key_version: "encryptionkey@0"
I20260808 16:59:40.962448 30929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:40.969744 30929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:40.971941 30929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:40.972090 30929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:40.972321 30929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:40.972764 31007 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 16:59:40.974648 30929 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 16:59:40.974705 30929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 16:59:40.974747 30929 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 16:59:40.975441 30929 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 16:59:40.975494 30929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 16:59:40.975564 31007 tablet_bootstrap.cc:492] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd: Bootstrap starting.
I20260808 16:59:40.981487 30929 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:33131
I20260808 16:59:40.981989 30929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:59:40.983031 31162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:33131 every 8 connection(s)
I20260808 16:59:40.986222 31007 log.cc:824] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:40.987741 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 30929
I20260808 16:59:40.988016 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:46293
--local_ip_for_outbound_sockets=127.8.14.66
--tserver_master_addrs=127.8.14.126:45175
--webserver_port=45707
--webserver_interface=127.8.14.66
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 16:59:40.989010 31007 tablet_bootstrap.cc:492] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 16:59:40.989355 31007 tablet_bootstrap.cc:492] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd: Bootstrap complete.
I20260808 16:59:40.990722 31007 ts_tablet_manager.cc:1397] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.014s
I20260808 16:59:40.992146 31007 raft_consensus.cc:348] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } }
I20260808 16:59:40.992525 31007 raft_consensus.cc:729] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb8fbeac543346de8fd0593d3f7b59dd, State: Initialized, Role: FOLLOWER
I20260808 16:59:40.992677 31007 consensus_queue.cc:260] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } }
I20260808 16:59:40.992950 31007 ts_tablet_manager.cc:1428] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260808 16:59:41.058008 31168 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 16:59:40.983546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:41761)
0808 16:59:40.986537 (+ 2991us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:40.986586 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:40.987168 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:41.057783 (+ 70615us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":2940,"thread_start_us":79,"threads_started":1}
W20260808 16:59:41.058301 31163 heartbeater.cc:644] Failed to heartbeat to 127.8.14.126:45175 (0 consecutive failures): Network error: Failed to ping master at 127.8.14.126:45175: Client connection negotiation failed: client connection to 127.8.14.126:45175: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 16:59:41.058691 31168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:41.058559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:37493)
0808 16:59:41.058613 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:41.058652 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 16:59:41.059183 31168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:41.059094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:46937)
0808 16:59:41.059139 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:41.059164 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 16:59:41.059314 31163 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 16:59:41.075577 31171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:41.075757 31171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:41.075778 31171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:41.076884 31171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:41.076924 31171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:41.076969 31171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:59:41.078493 31171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:46293
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=45707
--tserver_master_addrs=127.8.14.126:45175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:41.078716 31171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:41.078966 31171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:41.083889 31179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:41.083882 31180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:41.083894 31183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:41.084178 31171 server_base.cc:1034] running on GCE node
I20260808 16:59:41.084439 31171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:41.084681 31171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:41.085835 31171 hybrid_clock.cc:648] HybridClock initialized: now 1786208381085818 us; error 30 us; skew 500 ppm
Aug 08 16:59:41 dist-test-slave-1nl6 krb5kdc[29238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208381, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:41.088445 31171 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:59:41.088824 31171 webserver.cc:460] Webserver started at http://127.8.14.66:45707/ using document root <none> and password file <none>
I20260808 16:59:41.089008 31171 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:41.089051 31171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:41.090410 31171 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 16:59:41.090983 31198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:41.091140 31171 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 16:59:41.091229 31171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "e877162b46d64529850b1dc1a0bd79a2"
format_stamp: "Formatted at 2026-08-08 16:59:39 on dist-test-slave-1nl6"
server_key: "d266bbf42be5e7216cd2e10cae12e5d4"
server_key_iv: "823efe4ffdacad2f9314dd495b6f6b7d"
server_key_version: "encryptionkey@0"
I20260808 16:59:41.091544 31171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:41.098268 31171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:41.100028 31171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:41.100132 31171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:41.100307 31171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:41.100545 31171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:59:41.100576 31171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:41.100595 31171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:59:41.100608 31171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:41.106484 31171 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:46293
I20260808 16:59:41.106556 31365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:46293 every 8 connection(s)
I20260808 16:59:41.106874 31171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:59:41.107929 31369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:41.107586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:42331)
0808 16:59:41.107730 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:41.107800 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
W20260808 16:59:41.108321 31366 heartbeater.cc:644] Failed to heartbeat to 127.8.14.126:45175 (0 consecutive failures): Network error: Failed to ping master at 127.8.14.126:45175: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
I20260808 16:59:41.108579 31369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:41.108495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:41241)
0808 16:59:41.108535 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:41.108561 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 16:59:41.108842 31369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:41.108760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:42599)
0808 16:59:41.108808 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:41.108829 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 16:59:41.108955 31366 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 16:59:41.115953 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 31171
I20260808 16:59:41.116201 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:35721
--local_ip_for_outbound_sockets=127.8.14.67
--tserver_master_addrs=127.8.14.126:45175
--webserver_port=39041
--webserver_interface=127.8.14.67
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:59:41.209033 31370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:41.209249 31370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:41.209287 31370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:41.210433 31370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:41.210494 31370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:41.210539 31370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:59:41.212046 31370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:35721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=39041
--tserver_master_addrs=127.8.14.126:45175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:41.212312 31370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:41.212538 31370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:41.217496 31380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:41.217444 31381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:41.217522 31370 server_base.cc:1034] running on GCE node
W20260808 16:59:41.217439 31384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:41.217882 31370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:41.218087 31370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:41.219221 31370 hybrid_clock.cc:648] HybridClock initialized: now 1786208381219196 us; error 41 us; skew 500 ppm
Aug 08 16:59:41 dist-test-slave-1nl6 krb5kdc[29238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208381, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:41.221613 31370 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:59:41.222056 31370 webserver.cc:460] Webserver started at http://127.8.14.67:39041/ using document root <none> and password file <none>
I20260808 16:59:41.222302 31370 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:41.222352 31370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:41.223711 31370 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:41.224287 31392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:41.224460 31370 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:59:41.224550 31370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "62a5955432af4f1ebd619d258ec5e829"
format_stamp: "Formatted at 2026-08-08 16:59:39 on dist-test-slave-1nl6"
server_key: "0b155a473ddb646aee7bfcfef1d57eae"
server_key_iv: "483beff439602a9fd461293f03f7ba3e"
server_key_version: "encryptionkey@0"
I20260808 16:59:41.224860 31370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:41.242187 31370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:41.244047 31370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:41.244206 31370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:41.244450 31370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:41.244740 31370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:59:41.244774 31370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:41.244796 31370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:59:41.244860 31370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:41.250388 31370 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:35721
I20260808 16:59:41.250479 31558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:35721 every 8 connection(s)
I20260808 16:59:41.250782 31370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 16:59:41.251849 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 31370
I20260808 16:59:41.251904 31562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:41.251471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:56565)
0808 16:59:41.251612 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:41.251705 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
W20260808 16:59:41.252353 31559 heartbeater.cc:644] Failed to heartbeat to 127.8.14.126:45175 (0 consecutive failures): Network error: Failed to ping master at 127.8.14.126:45175: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
I20260808 16:59:41.252720 31564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:41.252543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:49835)
0808 16:59:41.252657 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:41.252689 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
I20260808 16:59:41.253032 31564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:41.252957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:35801)
0808 16:59:41.252994 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:41.253017 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 16:59:41.253159 31559 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 16:59:42.059854 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:42.059631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:53587)
0808 16:59:42.059748 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:42.059788 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
I20260808 16:59:42.109550 31566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:42.109243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:58369)
0808 16:59:42.109463 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:42.109501 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":67,"threads_started":1}
I20260808 16:59:42.253628 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:42.253434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:38991)
0808 16:59:42.253545 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:42.253577 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260808 16:59:42.804481 31568 raft_consensus.cc:482] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:59:42.804605 31568 raft_consensus.cc:504] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } }
I20260808 16:59:42.804905 31568 leader_election.cc:288] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721), e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66:46293)
Aug 08 16:59:42 dist-test-slave-1nl6 krb5kdc[29238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208380, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.67@KRBTEST.COM
Aug 08 16:59:42 dist-test-slave-1nl6 krb5kdc[29238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208380, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.66@KRBTEST.COM
I20260808 16:59:42.816460 31571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:42.805178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:44451 (local address 127.8.14.66:46293)
0808 16:59:42.805400 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:42.805411 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:42.805432 (+ 21us) server_negotiation.cc:407] Connection header received
0808 16:59:42.805877 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:42.805879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:42.805919 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:42.806067 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:42.806851 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:42.807781 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:42.808872 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:42.809257 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:42.809391 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:42.811993 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:42.811998 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:42.811998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:42.812012 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:42.814227 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:42.814847 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:42.814850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:42.814851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:42.814993 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:42.815366 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:42.815368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:42.815368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:42.815723 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:42.815837 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:42.816198 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 16:59:42.816347 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":65,"threads_started":1}
I20260808 16:59:42.816460 31572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:42.805280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:42797 (local address 127.8.14.67:35721)
0808 16:59:42.805416 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:42.805424 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:42.805442 (+ 18us) server_negotiation.cc:407] Connection header received
0808 16:59:42.805877 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:42.805879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:42.805919 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:42.806054 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:42.806871 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:42.807780 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:42.808856 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:42.809256 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:42.809380 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:42.812005 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:42.812009 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:42.812009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:42.812019 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:42.814205 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:42.814847 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:42.814850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:42.814851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:42.814993 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:42.815366 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:42.815368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:42.815368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:42.815705 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:42.815819 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:42.816198 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 16:59:42.816342 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260808 16:59:42.816524 31569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:42.805070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:35721 (local address 127.8.14.65:42797)
0808 16:59:42.805181 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:42.805192 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:42.805251 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:42.806157 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:42.806158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:42.806535 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:42.806723 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:42.806729 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:42.807895 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:42.807895 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:42.808752 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:42.808755 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:42.809353 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:42.809354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:42.809478 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:42.810118 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:42.810122 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:42.811605 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:42.814307 (+ 2702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:42.814318 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:42.814320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:42.814752 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:42.815078 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:42.815079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:42.815080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:42.815262 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:42.815800 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:42.815818 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:42.815991 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 16:59:42.816115 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 16:59:42.816377 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":170,"thread_start_us":52,"threads_started":1}
I20260808 16:59:42.817271 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:42.805070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:46293 (local address 127.8.14.65:44451)
0808 16:59:42.805196 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:42.805205 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:42.805251 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:42.806126 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:42.806129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:42.806536 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:42.806723 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:42.806728 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:42.807909 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:42.807910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:42.808751 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:42.808753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:42.809362 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:42.809363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:42.809478 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:42.810067 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:42.810075 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:42.811886 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:42.814307 (+ 2421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:42.814318 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:42.814320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:42.814752 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:42.815078 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:42.815079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:42.815080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:42.815262 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:42.815833 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:42.815833 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:42.815991 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 16:59:42.816115 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 16:59:42.816319 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":71,"thread_start_us":68,"threads_started":1}
W20260808 16:59:42.817389 31001 leader_election.cc:341] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721): Not found: Tablet not found: 4f2d4cb3b3564d3195557116341d25e7
W20260808 16:59:42.818037 30996 leader_election.cc:341] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66:46293): Not found: Tablet not found: 4f2d4cb3b3564d3195557116341d25e7
I20260808 16:59:42.818097 30996 leader_election.cc:302] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb8fbeac543346de8fd0593d3f7b59dd; no voters: 62a5955432af4f1ebd619d258ec5e829, e877162b46d64529850b1dc1a0bd79a2
I20260808 16:59:42.818193 31568 raft_consensus.cc:2726] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:59:43.060446 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:43.060310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:56073)
0808 16:59:43.060365 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:43.060402 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 16:59:43.110188 31574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:43.109975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:57907)
0808 16:59:43.110098 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:43.110144 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260808 16:59:43.254287 31575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:43.254049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:39439)
0808 16:59:43.254202 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:43.254239 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260808 16:59:44.061085 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:44.060846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:36235)
0808 16:59:44.060999 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:44.061037 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":64,"threads_started":1}
I20260808 16:59:44.110961 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:44.110686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:34007)
0808 16:59:44.110855 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:44.110886 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":106,"threads_started":1}
I20260808 16:59:44.254956 31581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:44.254729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:60495)
0808 16:59:44.254843 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:44.254877 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":49,"threads_started":1}
I20260808 16:59:44.970511 31582 raft_consensus.cc:482] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:59:44.970624 31582 raft_consensus.cc:504] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } }
I20260808 16:59:44.970788 31582 leader_election.cc:288] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721), e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66:46293)
W20260808 16:59:44.971280 30996 leader_election.cc:341] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66:46293): Not found: Tablet not found: 4f2d4cb3b3564d3195557116341d25e7
W20260808 16:59:44.971362 31001 leader_election.cc:341] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721): Not found: Tablet not found: 4f2d4cb3b3564d3195557116341d25e7
I20260808 16:59:44.971400 31001 leader_election.cc:302] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb8fbeac543346de8fd0593d3f7b59dd; no voters: 62a5955432af4f1ebd619d258ec5e829, e877162b46d64529850b1dc1a0bd79a2
I20260808 16:59:44.971483 31582 raft_consensus.cc:2726] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:59:45.061825 31583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:45.061574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:38207)
0808 16:59:45.061722 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:45.061759 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260808 16:59:45.111569 31584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:45.111378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:59165)
0808 16:59:45.111497 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:45.111530 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260808 16:59:45.255622 31585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:45.255361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:34461)
0808 16:59:45.255515 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:45.255557 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
I20260808 16:59:46.062762 31588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:46.062521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:58041)
0808 16:59:46.062671 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:46.062711 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":64,"threads_started":1}
I20260808 16:59:46.112263 31589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:46.112032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:53219)
0808 16:59:46.112179 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:46.112221 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":62,"threads_started":1}
I20260808 16:59:46.256250 31590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:46.256037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:33279)
0808 16:59:46.256164 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:46.256208 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
I20260808 16:59:47.063459 31591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:47.063252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:48555)
0808 16:59:47.063374 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:47.063410 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260808 16:59:47.112936 31592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:47.112726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:39647)
0808 16:59:47.112856 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:47.112893 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":60,"threads_started":1}
I20260808 16:59:47.256853 31595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:47.256646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:45023)
0808 16:59:47.256779 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:47.256811 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260808 16:59:47.802716 31597 raft_consensus.cc:482] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:59:47.802819 31597 raft_consensus.cc:504] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } }
I20260808 16:59:47.803004 31597 leader_election.cc:288] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721), e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66:46293)
W20260808 16:59:47.803443 30996 leader_election.cc:341] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66:46293): Not found: Tablet not found: 4f2d4cb3b3564d3195557116341d25e7
W20260808 16:59:47.803531 31001 leader_election.cc:341] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721): Not found: Tablet not found: 4f2d4cb3b3564d3195557116341d25e7
I20260808 16:59:47.803596 31001 leader_election.cc:302] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb8fbeac543346de8fd0593d3f7b59dd; no voters: 62a5955432af4f1ebd619d258ec5e829, e877162b46d64529850b1dc1a0bd79a2
I20260808 16:59:47.803702 31597 raft_consensus.cc:2726] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:59:48.064198 31599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:48.063986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:33287)
0808 16:59:48.064111 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:48.064148 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260808 16:59:48.113583 31600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:48.113383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:60413)
0808 16:59:48.113505 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:48.113541 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
I20260808 16:59:48.257560 31601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:48.257328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:54763)
0808 16:59:48.257472 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:48.257510 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260808 16:59:49.064917 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:49.064663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:44137)
0808 16:59:49.064831 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:49.064869 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":108,"threads_started":1}
I20260808 16:59:49.114217 31606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:49.113974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:46687)
0808 16:59:49.114142 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:49.114178 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":60,"threads_started":1}
I20260808 16:59:49.259619 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:49.259420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:49033)
0808 16:59:49.259527 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:49.259562 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":62,"threads_started":1}
I20260808 16:59:50.065640 31609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:50.065407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:56479)
0808 16:59:50.065553 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:50.065590 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260808 16:59:50.114861 31610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:50.114652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:54927)
0808 16:59:50.114776 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:50.114816 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260808 16:59:50.148869 31611 raft_consensus.cc:482] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:59:50.148988 31611 raft_consensus.cc:504] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } }
I20260808 16:59:50.149170 31611 leader_election.cc:288] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721), e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66:46293)
W20260808 16:59:50.149544 30996 leader_election.cc:341] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66:46293): Not found: Tablet not found: 4f2d4cb3b3564d3195557116341d25e7
W20260808 16:59:50.149623 31001 leader_election.cc:341] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721): Not found: Tablet not found: 4f2d4cb3b3564d3195557116341d25e7
I20260808 16:59:50.149654 30996 leader_election.cc:302] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb8fbeac543346de8fd0593d3f7b59dd; no voters: 62a5955432af4f1ebd619d258ec5e829, e877162b46d64529850b1dc1a0bd79a2
I20260808 16:59:50.149730 31611 raft_consensus.cc:2726] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:59:50.260349 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:50.260100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:60151)
0808 16:59:50.260247 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:50.260286 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260808 16:59:51.066316 31615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:51.066073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:40711)
0808 16:59:51.066199 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:51.066256 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260808 16:59:51.115583 31616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:51.115354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:36111)
0808 16:59:51.115502 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:51.115539 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260808 16:59:51.261056 31618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:51.260814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:38199)
0808 16:59:51.260973 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:51.261011 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":82,"threads_started":1}
I20260808 16:59:52.066977 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:52.066722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:34215)
0808 16:59:52.066846 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:52.066881 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260808 16:59:52.116253 31622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:52.116009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:54155)
0808 16:59:52.116172 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:52.116207 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":71,"threads_started":1}
I20260808 16:59:52.261802 31623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:52.261569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:59117)
0808 16:59:52.261719 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:52.261756 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260808 16:59:53.067629 31624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:53.067424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:40285)
0808 16:59:53.067544 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:53.067581 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260808 16:59:53.117142 31625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:53.116934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:56411)
0808 16:59:53.117060 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:53.117096 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
I20260808 16:59:53.202833 31626 raft_consensus.cc:482] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:59:53.203126 31626 raft_consensus.cc:504] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } }
I20260808 16:59:53.203357 31626 leader_election.cc:288] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721), e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66:46293)
W20260808 16:59:53.203750 31001 leader_election.cc:341] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721): Not found: Tablet not found: 4f2d4cb3b3564d3195557116341d25e7
W20260808 16:59:53.203841 30996 leader_election.cc:341] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66:46293): Not found: Tablet not found: 4f2d4cb3b3564d3195557116341d25e7
I20260808 16:59:53.203938 30996 leader_election.cc:302] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb8fbeac543346de8fd0593d3f7b59dd; no voters: 62a5955432af4f1ebd619d258ec5e829, e877162b46d64529850b1dc1a0bd79a2
I20260808 16:59:53.204012 31626 raft_consensus.cc:2726] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:59:53.262503 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:53.262287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:46985)
0808 16:59:53.262423 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:53.262458 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
I20260808 16:59:54.068362 31629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:54.068144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:38551)
0808 16:59:54.068271 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:54.068314 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260808 16:59:54.117961 31630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:54.117630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:59321)
0808 16:59:54.117872 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:54.117911 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":95,"threads_started":1}
I20260808 16:59:54.263338 31631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:54.263067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:48113)
0808 16:59:54.263223 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:54.263261 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":80,"threads_started":1}
I20260808 16:59:55.069034 31632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:55.068821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:33153)
0808 16:59:55.068952 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:55.068986 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260808 16:59:55.118736 31634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:55.118489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:34867)
0808 16:59:55.118648 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:55.118687 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260808 16:59:55.264025 31635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:55.263814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:42251)
0808 16:59:55.263942 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:55.263979 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260808 16:59:56.069792 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:56.069583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.65:44289)
0808 16:59:56.069708 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:56.069745 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260808 16:59:56.119741 31637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:56.119519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.66:42631)
0808 16:59:56.119663 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:56.119697 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
I20260808 16:59:56.172550 31640 raft_consensus.cc:482] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:59:56.172664 31640 raft_consensus.cc:504] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62a5955432af4f1ebd619d258ec5e829" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 35721 } } peers { permanent_uuid: "e877162b46d64529850b1dc1a0bd79a2" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 46293 } } peers { permanent_uuid: "bb8fbeac543346de8fd0593d3f7b59dd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 33131 } }
I20260808 16:59:56.172881 31640 leader_election.cc:288] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721), e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66:46293)
W20260808 16:59:56.173219 31001 leader_election.cc:341] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 62a5955432af4f1ebd619d258ec5e829 (127.8.14.67:35721): Not found: Tablet not found: 4f2d4cb3b3564d3195557116341d25e7
W20260808 16:59:56.173339 30996 leader_election.cc:341] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e877162b46d64529850b1dc1a0bd79a2 (127.8.14.66:46293): Not found: Tablet not found: 4f2d4cb3b3564d3195557116341d25e7
I20260808 16:59:56.173381 30996 leader_election.cc:302] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb8fbeac543346de8fd0593d3f7b59dd; no voters: 62a5955432af4f1ebd619d258ec5e829, e877162b46d64529850b1dc1a0bd79a2
I20260808 16:59:56.173491 31640 raft_consensus.cc:2726] T 4f2d4cb3b3564d3195557116341d25e7 P bb8fbeac543346de8fd0593d3f7b59dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 16:59:56.261262 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 30929
I20260808 16:59:56.271608 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:56.267225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45175 (local address 127.8.14.67:58573)
0808 16:59:56.271507 (+ 4282us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:56.271548 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:45175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4263,"thread_start_us":709,"threads_started":1}
I20260808 16:59:56.280573 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 31171
I20260808 16:59:56.291218 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 31370
2026-08-08T16:59:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18789 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 16:59:56 dist-test-slave-1nl6 krb5kdc[31645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 16:59:56 dist-test-slave-1nl6 krb5kdc[31645](info): set up 2 sockets
Aug 08 16:59:56 dist-test-slave-1nl6 krb5kdc[31645](info): commencing operation
krb5kdc: starting...
W20260808 16:59:57.517973 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 16:59:57 dist-test-slave-1nl6 krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T16:59:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T16:59:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 16:59:57.643524 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:46633
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:46633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:59:57.726460 31666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:57.726672 31666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:57.726694 31666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:57.727963 31666 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 16:59:57.728008 31666 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 16:59:57.728020 31666 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 16:59:57.728032 31666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 16:59:57.728045 31666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:57.728058 31666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:57.728070 31666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 16:59:57.728081 31666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 16:59:57.729519 31666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:46633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:46633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:57.729722 31666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:57.729898 31666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:57.734833 31674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:57.734854 31673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:57.734841 31676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:57.734923 31666 server_base.cc:1034] running on GCE node
I20260808 16:59:57.735369 31666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:57.735690 31666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:57.736862 31666 hybrid_clock.cc:648] HybridClock initialized: now 1786208397736856 us; error 37 us; skew 500 ppm
Aug 08 16:59:57 dist-test-slave-1nl6 krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208397, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:57.739444 31666 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 16:59:57.739905 31666 webserver.cc:460] Webserver started at http://127.8.14.126:35963/ using document root <none> and password file <none>
I20260808 16:59:57.740175 31666 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:57.740253 31666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:57.740492 31666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:59:57.741525 31666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "5e847c66e5364aed8144db9949cd8fca"
format_stamp: "Formatted at 2026-08-08 16:59:57 on dist-test-slave-1nl6"
server_key: "d71e8633554e6e1edfbd41924581cc00"
server_key_iv: "ddb9bd47d641f8aa1bcc42a9cfc4737b"
server_key_version: "encryptionkey@0"
I20260808 16:59:57.741854 31666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "5e847c66e5364aed8144db9949cd8fca"
format_stamp: "Formatted at 2026-08-08 16:59:57 on dist-test-slave-1nl6"
server_key: "d71e8633554e6e1edfbd41924581cc00"
server_key_iv: "ddb9bd47d641f8aa1bcc42a9cfc4737b"
server_key_version: "encryptionkey@0"
I20260808 16:59:57.743180 31666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 16:59:57.743877 31686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:57.744053 31666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:57.744158 31666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "5e847c66e5364aed8144db9949cd8fca"
format_stamp: "Formatted at 2026-08-08 16:59:57 on dist-test-slave-1nl6"
server_key: "d71e8633554e6e1edfbd41924581cc00"
server_key_iv: "ddb9bd47d641f8aa1bcc42a9cfc4737b"
server_key_version: "encryptionkey@0"
I20260808 16:59:57.744246 31666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:57.750617 31666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:57.752465 31666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:57.752628 31666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:57.756826 31666 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:46633
I20260808 16:59:57.756860 31757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:46633 every 8 connection(s)
I20260808 16:59:57.757239 31666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 16:59:57.758220 31758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:57.758391 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 31666
I20260808 16:59:57.758508 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 16:59:57.758729 8249 external_mini_cluster.cc:1428] Setting key fd34ac197f644434f5976bb86fabe62a
I20260808 16:59:57.761358 31758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: Bootstrap starting.
I20260808 16:59:57.762002 31758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:57.762295 31758 log.cc:824] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:57.763051 31758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: No bootstrap required, opened a new log
I20260808 16:59:57.765132 31758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } }
I20260808 16:59:57.765295 31758 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Aug 08 16:59:57 dist-test-slave-1nl6 krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:57.765348 31758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e847c66e5364aed8144db9949cd8fca, State: Initialized, Role: FOLLOWER
I20260808 16:59:57.765522 31758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } }
I20260808 16:59:57.765625 31758 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 16:59:57.765668 31758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 16:59:57.765715 31758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:57.766285 31758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } }
I20260808 16:59:57.766399 31758 leader_election.cc:302] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e847c66e5364aed8144db9949cd8fca; no voters:
I20260808 16:59:57.766572 31758 leader_election.cc:288] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 16:59:57.766640 31766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:59:57.766757 31766 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 1 LEADER]: Becoming Leader. State: Replica: 5e847c66e5364aed8144db9949cd8fca, State: Running, Role: LEADER
I20260808 16:59:57.766885 31758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [sys.catalog]: configured and running, proceeding with master startup.
I20260808 16:59:57.767262 31766 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } }
I20260808 16:59:57.768877 31768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e847c66e5364aed8144db9949cd8fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } } }
I20260808 16:59:57.769265 31768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [sys.catalog]: This master's current role is: LEADER
W20260808 16:59:57.769286 31788 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 16:59:57.769330 31788 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 16:59:57.769452 31792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 16:59:57.768890 31767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e847c66e5364aed8144db9949cd8fca. Latest consensus state: current_term: 1 leader_uuid: "5e847c66e5364aed8144db9949cd8fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } } }
I20260808 16:59:57.769551 31767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [sys.catalog]: This master's current role is: LEADER
I20260808 16:59:57.769754 31792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 16:59:57.770563 31764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:57.759311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47956 (local address 127.8.14.126:46633)
0808 16:59:57.759512 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:57.759521 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:57.759555 (+ 34us) server_negotiation.cc:407] Connection header received
0808 16:59:57.760012 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:57.760019 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:57.760066 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:57.760217 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:57.760881 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:57.761869 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:57.762776 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:57.763304 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:57.763436 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:57.765922 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:57.765926 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:57.765927 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:57.765944 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:57.768152 (+ 2208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:57.768594 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:57.768597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:57.768597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:57.768732 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:57.769172 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:57.769173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:57.769173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:57.769499 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:57.769673 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:57.770201 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 16:59:57.770374 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":80,"threads_started":1}
I20260808 16:59:57.771490 31792 catalog_manager.cc:1347] Generated new cluster ID: 3635b83ffb8c4c09aa883ac513475ce1
I20260808 16:59:57.771545 31792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 16:59:57.796180 31792 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 16:59:57.796707 31792 catalog_manager.cc:1504] Loading token signing keys...
I20260808 16:59:57.808203 31792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: Generated new TSK 0
I20260808 16:59:57.808396 31792 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 16:59:57.838865 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46633
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:59:57.920938 31800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:57.921139 31800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:57.921160 31800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:57.922295 31800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:57.922338 31800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:57.922363 31800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 16:59:57.923933 31800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:57.924175 31800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:57.924430 31800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:57.929371 31814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:57.929373 31807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:57.929517 31800 server_base.cc:1034] running on GCE node
W20260808 16:59:57.929437 31808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:57.929832 31800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:57.930075 31800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:57.931222 31800 hybrid_clock.cc:648] HybridClock initialized: now 1786208397931209 us; error 29 us; skew 500 ppm
Aug 08 16:59:57 dist-test-slave-1nl6 krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208397, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:57.933662 31800 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 16:59:57.934031 31800 webserver.cc:460] Webserver started at http://127.8.14.65:35417/ using document root <none> and password file <none>
I20260808 16:59:57.934223 31800 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:57.934263 31800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:57.934353 31800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:59:57.935405 31800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "25d053f330d747329f158333e7295ac7"
format_stamp: "Formatted at 2026-08-08 16:59:57 on dist-test-slave-1nl6"
server_key: "5d209666db4cb765e5e26b5633305c43"
server_key_iv: "0b7d8a4a4bebc4d6fc3562762dab6c72"
server_key_version: "encryptionkey@0"
I20260808 16:59:57.935693 31800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "25d053f330d747329f158333e7295ac7"
format_stamp: "Formatted at 2026-08-08 16:59:57 on dist-test-slave-1nl6"
server_key: "5d209666db4cb765e5e26b5633305c43"
server_key_iv: "0b7d8a4a4bebc4d6fc3562762dab6c72"
server_key_version: "encryptionkey@0"
I20260808 16:59:57.936942 31800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 16:59:57.937624 31822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:57.937822 31800 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 16:59:57.937903 31800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "25d053f330d747329f158333e7295ac7"
format_stamp: "Formatted at 2026-08-08 16:59:57 on dist-test-slave-1nl6"
server_key: "5d209666db4cb765e5e26b5633305c43"
server_key_iv: "0b7d8a4a4bebc4d6fc3562762dab6c72"
server_key_version: "encryptionkey@0"
I20260808 16:59:57.937969 31800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:57.954137 31800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:57.956048 31800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:57.956167 31800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:57.956341 31800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:57.956576 31800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:59:57.956606 31800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:57.956627 31800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:59:57.956645 31800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:57.962273 31800 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:45261
I20260808 16:59:57.962339 31966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:45261 every 8 connection(s)
I20260808 16:59:57.962577 31800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 16:59:57.964069 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 31800
I20260808 16:59:57.964195 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 16:59:57.964469 8249 external_mini_cluster.cc:1428] Setting key 770abc4cf1669d4fcfc8417c191a7669
Aug 08 16:59:57 dist-test-slave-1nl6 krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208397, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:57.973436 31764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:57.963344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:34883 (local address 127.8.14.126:46633)
0808 16:59:57.963411 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:57.963413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:57.963643 (+ 230us) server_negotiation.cc:407] Connection header received
0808 16:59:57.964361 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:57.964362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:57.964373 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:57.964423 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:57.965588 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:57.966083 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:57.967623 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:57.967988 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:57.968058 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:57.970545 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:57.970547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:57.970547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:57.970555 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:57.972010 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:57.972462 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:57.972464 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:57.972464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:57.972525 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:57.972780 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:57.972781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:57.972781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:57.972895 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:57.972974 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:57.973243 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 16:59:57.973349 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 16:59:57.973603 31972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:57.963314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:34883)
0808 16:59:57.963503 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:57.963547 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:57.964207 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:57.964502 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:57.964506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:57.965171 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:57.965488 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:57.965495 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:57.966181 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:57.966183 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:57.967538 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:57.967541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:57.968084 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:57.968085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:57.968263 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:57.968749 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:57.968761 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:57.970443 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:57.972124 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:57.972132 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:57.972133 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:57.972370 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:57.972596 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:57.972597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:57.972597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:57.972708 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:57.972966 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:57.972973 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:57.973111 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 16:59:57.973220 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 16:59:57.973380 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
I20260808 16:59:57.974040 31968 heartbeater.cc:347] Connected to a master server at 127.8.14.126:46633
I20260808 16:59:57.974140 31968 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:57.974310 31968 heartbeater.cc:511] Master 127.8.14.126:46633 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
I20260808 16:59:57.974859 31713 ts_manager.cc:194] Registered new tserver with Master: 25d053f330d747329f158333e7295ac7 (127.8.14.65:45261)
I20260808 16:59:57.975647 31713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:34883
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 16:59:57.991549 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46633
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:59:58.074608 31976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:58.074815 31976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:58.074837 31976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:58.075976 31976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:58.076020 31976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:58.076046 31976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 16:59:58.077453 31976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:58.077634 31976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:58.077813 31976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:58.082792 31981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:58.082765 31984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:58.082810 31976 server_base.cc:1034] running on GCE node
W20260808 16:59:58.082782 31982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:58.083204 31976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:58.083448 31976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:58.084611 31976 hybrid_clock.cc:648] HybridClock initialized: now 1786208398084600 us; error 45 us; skew 500 ppm
Aug 08 16:59:58 dist-test-slave-1nl6 krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208398, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:58.087086 31976 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 16:59:58.087520 31976 webserver.cc:460] Webserver started at http://127.8.14.66:33525/ using document root <none> and password file <none>
I20260808 16:59:58.087745 31976 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:58.087815 31976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:58.087970 31976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:59:58.088989 31976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "a6d4e9cdda2240949054f7005002891c"
format_stamp: "Formatted at 2026-08-08 16:59:58 on dist-test-slave-1nl6"
server_key: "98d2ef0974494be6d4df7656980c2920"
server_key_iv: "dba150f6d4f7d2544d79cb9162b46d17"
server_key_version: "encryptionkey@0"
I20260808 16:59:58.089358 31976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "a6d4e9cdda2240949054f7005002891c"
format_stamp: "Formatted at 2026-08-08 16:59:58 on dist-test-slave-1nl6"
server_key: "98d2ef0974494be6d4df7656980c2920"
server_key_iv: "dba150f6d4f7d2544d79cb9162b46d17"
server_key_version: "encryptionkey@0"
I20260808 16:59:58.090679 31976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 16:59:58.091522 31996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:58.091701 31976 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:58.091809 31976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "a6d4e9cdda2240949054f7005002891c"
format_stamp: "Formatted at 2026-08-08 16:59:58 on dist-test-slave-1nl6"
server_key: "98d2ef0974494be6d4df7656980c2920"
server_key_iv: "dba150f6d4f7d2544d79cb9162b46d17"
server_key_version: "encryptionkey@0"
I20260808 16:59:58.091912 31976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:58.123809 31976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:58.125774 31976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:58.125950 31976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:58.126227 31976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:58.126559 31976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:59:58.126616 31976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:58.126652 31976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:59:58.126682 31976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:58.132789 31976 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:34407
I20260808 16:59:58.132843 32157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:34407 every 8 connection(s)
I20260808 16:59:58.133247 31976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 16:59:58.136433 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 31976
I20260808 16:59:58.136569 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 16:59:58.136799 8249 external_mini_cluster.cc:1428] Setting key b2f8c5235e6361ccfef55c7cb226030a
Aug 08 16:59:58 dist-test-slave-1nl6 krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208398, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:58.144869 31764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:58.133983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:58209 (local address 127.8.14.126:46633)
0808 16:59:58.134047 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:58.134049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:58.134285 (+ 236us) server_negotiation.cc:407] Connection header received
0808 16:59:58.134825 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:58.134826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:58.134840 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:58.134887 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:58.135875 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.136405 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.137800 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.138165 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.138227 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:58.141244 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:58.141246 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:58.141257 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:58.141263 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:58.143259 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:58.143701 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:58.143702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:58.143703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:58.143766 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:58.144062 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:58.144063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:58.144063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:58.144241 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:58.144321 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:58.144693 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 16:59:58.144780 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 16:59:58.145047 32160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:58.133963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:58209)
0808 16:59:58.134133 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:58.134194 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:58.134703 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:58.134997 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:58.135002 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:58.135566 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:58.135755 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:58.135759 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.136503 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.136505 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:58.137712 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:58.137716 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.138278 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.138279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:58.138482 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:58.139124 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:58.139134 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:58.141131 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:58.143347 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:58.143355 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:58.143357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:58.143603 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:58.143839 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:58.143840 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:58.143841 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:58.143973 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:58.144329 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:58.144336 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:58.144499 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 16:59:58.144627 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 16:59:58.144794 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":71,"threads_started":1}
I20260808 16:59:58.145813 32158 heartbeater.cc:347] Connected to a master server at 127.8.14.126:46633
I20260808 16:59:58.145922 32158 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:58.146129 32158 heartbeater.cc:511] Master 127.8.14.126:46633 requested a full tablet report, sending...
I20260808 16:59:58.146819 31713 ts_manager.cc:194] Registered new tserver with Master: a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407)
I20260808 16:59:58.147527 31713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:58209
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 16:59:58.166883 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46633
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 16:59:58.247117 32164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 16:59:58.247304 32164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 16:59:58.247326 32164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 16:59:58.248471 32164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 16:59:58.248510 32164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 16:59:58.248535 32164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 16:59:58.249893 32164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 16:59:58.250069 32164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 16:59:58.250250 32164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 16:59:58.255240 32171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:58.255293 32164 server_base.cc:1034] running on GCE node
W20260808 16:59:58.255188 32173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 16:59:58.255203 32175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 16:59:58.255577 32164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 16:59:58.255820 32164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 16:59:58.256959 32164 hybrid_clock.cc:648] HybridClock initialized: now 1786208398256934 us; error 42 us; skew 500 ppm
Aug 08 16:59:58 dist-test-slave-1nl6 krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208398, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 16:59:58.259526 32164 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 16:59:58.259922 32164 webserver.cc:460] Webserver started at http://127.8.14.67:43221/ using document root <none> and password file <none>
I20260808 16:59:58.260156 32164 fs_manager.cc:359] Metadata directory not provided
I20260808 16:59:58.260202 32164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 16:59:58.260347 32164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 16:59:58.261336 32164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "5c20d9c140184f969e86ad6d3b0bad77"
format_stamp: "Formatted at 2026-08-08 16:59:58 on dist-test-slave-1nl6"
server_key: "6dc565364b867e034a4db7585bc26e74"
server_key_iv: "9ae6da08c25e0ffaefd6edcfa5fba1b8"
server_key_version: "encryptionkey@0"
I20260808 16:59:58.261674 32164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "5c20d9c140184f969e86ad6d3b0bad77"
format_stamp: "Formatted at 2026-08-08 16:59:58 on dist-test-slave-1nl6"
server_key: "6dc565364b867e034a4db7585bc26e74"
server_key_iv: "9ae6da08c25e0ffaefd6edcfa5fba1b8"
server_key_version: "encryptionkey@0"
I20260808 16:59:58.263012 32164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 16:59:58.263782 32185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:58.264024 32164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 16:59:58.264111 32164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "5c20d9c140184f969e86ad6d3b0bad77"
format_stamp: "Formatted at 2026-08-08 16:59:58 on dist-test-slave-1nl6"
server_key: "6dc565364b867e034a4db7585bc26e74"
server_key_iv: "9ae6da08c25e0ffaefd6edcfa5fba1b8"
server_key_version: "encryptionkey@0"
I20260808 16:59:58.264186 32164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 16:59:58.287844 32164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 16:59:58.290144 32164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 16:59:58.290269 32164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 16:59:58.290438 32164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 16:59:58.290746 32164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 16:59:58.290784 32164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:58.290828 32164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 16:59:58.290854 32164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 16:59:58.297302 32164 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:45431
I20260808 16:59:58.297362 32341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:45431 every 8 connection(s)
I20260808 16:59:58.297684 32164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 16:59:58.302028 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 32164
I20260808 16:59:58.302165 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 16:59:58.302376 8249 external_mini_cluster.cc:1428] Setting key 47ef4f1c61ac542960679d7271e8445e
Aug 08 16:59:58 dist-test-slave-1nl6 krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208398, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 16:59:58.306807 31764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:58.298383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:33267 (local address 127.8.14.126:46633)
0808 16:59:58.298433 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:58.298436 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:58.298640 (+ 204us) server_negotiation.cc:407] Connection header received
0808 16:59:58.299136 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:58.299137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:58.299147 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:58.299208 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:58.300105 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.300609 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.301624 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.301969 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.302068 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:58.304367 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:58.304369 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:58.304369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:58.304376 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:58.305620 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:58.305992 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:58.305993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:58.305994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:58.306047 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:58.306267 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:58.306268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:58.306268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:58.306372 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:58.306427 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:58.306674 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 16:59:58.306741 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 16:59:58.306972 32344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:58.298346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:33267)
0808 16:59:58.298509 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:58.298551 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:58.299051 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:58.299259 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:58.299262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:58.299806 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0808 16:59:58.299994 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:58.299999 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.300719 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.300720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:58.301522 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:58.301525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.302071 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.302072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:58.302264 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:58.302793 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 16:59:58.302807 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 16:59:58.304269 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 16:59:58.305701 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:58.305707 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:58.305709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:58.305916 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:58.306103 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 16:59:58.306104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 16:59:58.306104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 16:59:58.306212 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 16:59:58.306427 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 16:59:58.306431 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 16:59:58.306554 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 16:59:58.306632 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 16:59:58.306749 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260808 16:59:58.307443 32342 heartbeater.cc:347] Connected to a master server at 127.8.14.126:46633
I20260808 16:59:58.307570 32342 heartbeater.cc:464] Registering TS with master...
I20260808 16:59:58.307768 32342 heartbeater.cc:511] Master 127.8.14.126:46633 requested a full tablet report, sending...
I20260808 16:59:58.308209 31713 ts_manager.cc:194] Registered new tserver with Master: 5c20d9c140184f969e86ad6d3b0bad77 (127.8.14.67:45431)
I20260808 16:59:58.308657 31713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:33267
I20260808 16:59:58.314075 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 16:59:58.321846 31764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:58.315513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47964 (local address 127.8.14.126:46633)
0808 16:59:58.315600 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:58.315603 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:58.315727 (+ 124us) server_negotiation.cc:407] Connection header received
0808 16:59:58.315811 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:58.315812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:58.315823 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:58.315900 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 16:59:58.316560 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.317074 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.317863 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.318103 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.318182 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:58.319439 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 16:59:58.319440 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 16:59:58.319440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 16:59:58.319449 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 16:59:58.320629 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:58.320999 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:58.321001 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:58.321001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:58.321064 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 16:59:58.321292 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 16:59:58.321293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 16:59:58.321293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 16:59:58.321439 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 16:59:58.321544 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:58.321660 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 16:59:58.321751 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 16:59:58.323259 31713 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 16:59:58.324764 31713 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 16:59:58.331262 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:58.327188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:45261 (local address 127.0.0.1:44814)
0808 16:59:58.327352 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:58.327362 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:58.327421 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:58.327799 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:58.327800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:58.327802 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:58.327873 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:58.327874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.328814 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.328815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:58.330056 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:58.330059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.330873 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.330874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:58.331026 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:58.331066 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 16:59:58.331123 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 16:59:58.331188 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":63,"thread_start_us":83,"threads_started":1}
I20260808 16:59:58.331437 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:58.326947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:45431 (local address 127.0.0.1:35838)
0808 16:59:58.327207 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:58.327245 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:58.327358 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:58.327663 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:58.327666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:58.327668 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:58.327767 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:58.327768 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.329116 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.329117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:58.330432 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:58.330434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.331162 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.331163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:58.331228 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:58.331233 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 16:59:58.331293 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 16:59:58.331336 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":103,"thread_start_us":61,"threads_started":1}
I20260808 16:59:58.331629 32363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:58.327439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44814 (local address 127.8.14.65:45261)
0808 16:59:58.327592 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:58.327596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:58.327605 (+ 9us) server_negotiation.cc:407] Connection header received
0808 16:59:58.327651 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:58.327652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:58.327670 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:58.327953 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:58.328035 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.328700 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.330144 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.330766 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.331393 (+ 627us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:58.331472 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:58.331499 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 16:59:58.331544 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260808 16:59:58.331872 32358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:58.327037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35838 (local address 127.8.14.67:45431)
0808 16:59:58.327181 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:58.327187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:58.327449 (+ 262us) server_negotiation.cc:407] Connection header received
0808 16:59:58.327493 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:58.327496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:58.327511 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:58.327582 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:58.327846 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.328996 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.330517 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.331070 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.331160 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:58.331219 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:58.331747 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 16:59:58.331793 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260808 16:59:58.332087 32362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:58.326962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:34407 (local address 127.0.0.1:42702)
0808 16:59:58.327335 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:58.327345 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:58.327395 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:58.327666 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:58.327667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:58.327668 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:58.327759 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:58.327760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.329116 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.329117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:58.330388 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:58.330391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.331603 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.331604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:58.331671 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:58.331675 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 16:59:58.331956 (+ 281us) client_negotiation.cc:240] Negotiation successful
0808 16:59:58.332008 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":40,"thread_start_us":89,"threads_started":1}
I20260808 16:59:58.332288 32360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:58.327093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42702 (local address 127.8.14.66:34407)
0808 16:59:58.327298 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:58.327303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:58.327461 (+ 158us) server_negotiation.cc:407] Connection header received
0808 16:59:58.327506 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:58.327508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:58.327522 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:58.327591 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:58.327860 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.328986 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.330483 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.331461 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.331523 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:58.331571 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:58.332137 (+ 566us) server_negotiation.cc:299] Negotiation successful
0808 16:59:58.332190 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":88,"threads_started":1}
I20260808 16:59:58.332404 31883 tablet_service.cc:1467] Processing CreateTablet for tablet a485f49b846b41ec9aa7f9ea79752345 (DEFAULT_TABLE table=test-table [id=09d523fb05884270b5b81e4e8f0d7a02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:59:58.332808 31883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a485f49b846b41ec9aa7f9ea79752345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:58.332945 32066 tablet_service.cc:1467] Processing CreateTablet for tablet a485f49b846b41ec9aa7f9ea79752345 (DEFAULT_TABLE table=test-table [id=09d523fb05884270b5b81e4e8f0d7a02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:59:58.333003 32255 tablet_service.cc:1467] Processing CreateTablet for tablet a485f49b846b41ec9aa7f9ea79752345 (DEFAULT_TABLE table=test-table [id=09d523fb05884270b5b81e4e8f0d7a02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 16:59:58.333271 32066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a485f49b846b41ec9aa7f9ea79752345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:58.333359 32255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a485f49b846b41ec9aa7f9ea79752345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 16:59:58.336001 32365 tablet_bootstrap.cc:492] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77: Bootstrap starting.
I20260808 16:59:58.336651 32366 tablet_bootstrap.cc:492] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: Bootstrap starting.
I20260808 16:59:58.336701 32368 tablet_bootstrap.cc:492] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c: Bootstrap starting.
I20260808 16:59:58.336715 32365 tablet_bootstrap.cc:654] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:58.337244 32366 tablet_bootstrap.cc:654] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:58.337381 32365 log.cc:824] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:58.337430 32368 tablet_bootstrap.cc:654] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c: Neither blocks nor log segments found. Creating new log.
I20260808 16:59:58.337898 32366 log.cc:824] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:58.337972 32368 log.cc:824] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c: Log is configured to *not* fsync() on all Append() calls
I20260808 16:59:58.338248 32365 tablet_bootstrap.cc:492] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77: No bootstrap required, opened a new log
I20260808 16:59:58.338322 32365 ts_tablet_manager.cc:1397] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:59:58.338578 32366 tablet_bootstrap.cc:492] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: No bootstrap required, opened a new log
I20260808 16:59:58.338630 32366 ts_tablet_manager.cc:1397] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:59:58.339427 32368 tablet_bootstrap.cc:492] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c: No bootstrap required, opened a new log
I20260808 16:59:58.339504 32368 ts_tablet_manager.cc:1397] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 16:59:58.340255 32365 raft_consensus.cc:348] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } } peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } }
I20260808 16:59:58.340440 32365 raft_consensus.cc:374] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:58.340332 32366 raft_consensus.cc:348] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } } peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } }
I20260808 16:59:58.340481 32365 raft_consensus.cc:729] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c20d9c140184f969e86ad6d3b0bad77, State: Initialized, Role: FOLLOWER
I20260808 16:59:58.340487 32366 raft_consensus.cc:374] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:58.340523 32366 raft_consensus.cc:729] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25d053f330d747329f158333e7295ac7, State: Initialized, Role: FOLLOWER
I20260808 16:59:58.340622 32366 consensus_queue.cc:260] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } } peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } }
I20260808 16:59:58.340617 32365 consensus_queue.cc:260] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } } peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } }
I20260808 16:59:58.340930 32365 ts_tablet_manager.cc:1428] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:59:58.340929 32366 ts_tablet_manager.cc:1428] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 16:59:58.341151 32342 heartbeater.cc:503] Master 127.8.14.126:46633 was elected leader, sending a full tablet report...
I20260808 16:59:58.341496 31968 heartbeater.cc:503] Master 127.8.14.126:46633 was elected leader, sending a full tablet report...
I20260808 16:59:58.341724 32368 raft_consensus.cc:348] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } } peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } }
I20260808 16:59:58.341883 32368 raft_consensus.cc:374] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 16:59:58.341928 32368 raft_consensus.cc:729] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6d4e9cdda2240949054f7005002891c, State: Initialized, Role: FOLLOWER
I20260808 16:59:58.342046 32368 consensus_queue.cc:260] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } } peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } }
I20260808 16:59:58.342334 32368 ts_tablet_manager.cc:1428] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 16:59:58.342334 32158 heartbeater.cc:503] Master 127.8.14.126:46633 was elected leader, sending a full tablet report...
W20260808 16:59:58.383845 32159 tablet.cc:2369] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 16:59:58.463289 31969 tablet.cc:2369] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 16:59:58.487378 32374 raft_consensus.cc:482] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 16:59:58.487543 32374 raft_consensus.cc:504] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } } peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } }
I20260808 16:59:58.487972 32374 leader_election.cc:288] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25d053f330d747329f158333e7295ac7 (127.8.14.65:45261), a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407)
I20260808 16:59:58.490741 32344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:58.488148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:45261 (local address 127.8.14.67:52231)
0808 16:59:58.488264 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:58.488276 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:58.488342 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:58.488497 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:58.488498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:58.488500 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:58.488573 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:58.488574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.489211 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.489212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:58.489934 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:58.489936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.490478 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.490479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:58.490557 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:58.490560 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:59:58.490628 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 16:59:58.490667 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 16:59:58.490741 32363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:58.488319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:52231 (local address 127.8.14.65:45261)
0808 16:59:58.488367 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:58.488369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:58.488379 (+ 10us) server_negotiation.cc:407] Connection header received
0808 16:59:58.488407 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:58.488408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:58.488419 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:58.488490 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:58.488660 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.489126 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.490032 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.490392 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.490477 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:58.490488 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:58.490634 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 16:59:58.490675 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 16:59:58.490957 32360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:58.488281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:60673 (local address 127.8.14.66:34407)
0808 16:59:58.488334 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:58.488336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:58.488485 (+ 149us) server_negotiation.cc:407] Connection header received
0808 16:59:58.488579 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:58.488580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:58.488590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:58.488661 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 16:59:58.488819 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.489269 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.490200 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.490585 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.490670 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:58.490680 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:58.490831 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 16:59:58.490871 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 16:59:58.490974 32377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:58.488202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:34407 (local address 127.8.14.67:60673)
0808 16:59:58.488414 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:58.488422 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 16:59:58.488503 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 16:59:58.488676 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 16:59:58.488677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 16:59:58.488678 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 16:59:58.488751 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:58.488752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.489362 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.489363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:58.490103 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 16:59:58.490105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.490679 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.490680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 16:59:58.490754 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:58.490757 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 16:59:58.490826 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 16:59:58.490864 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":11,"thread_start_us":144,"threads_started":1}
I20260808 16:59:58.491282 31911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a485f49b846b41ec9aa7f9ea79752345" candidate_uuid: "5c20d9c140184f969e86ad6d3b0bad77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25d053f330d747329f158333e7295ac7" is_pre_election: true
I20260808 16:59:58.491384 32099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a485f49b846b41ec9aa7f9ea79752345" candidate_uuid: "5c20d9c140184f969e86ad6d3b0bad77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6d4e9cdda2240949054f7005002891c" is_pre_election: true
I20260808 16:59:58.491460 31911 raft_consensus.cc:2454] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c20d9c140184f969e86ad6d3b0bad77 in term 0.
I20260808 16:59:58.491503 32099 raft_consensus.cc:2454] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c20d9c140184f969e86ad6d3b0bad77 in term 0.
I20260808 16:59:58.491698 32189 leader_election.cc:302] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25d053f330d747329f158333e7295ac7, 5c20d9c140184f969e86ad6d3b0bad77; no voters:
I20260808 16:59:58.491858 32374 raft_consensus.cc:2781] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 16:59:58.491925 32374 raft_consensus.cc:482] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 16:59:58.491942 32374 raft_consensus.cc:3037] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:58.492710 32374 raft_consensus.cc:504] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } } peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } }
I20260808 16:59:58.492858 32374 leader_election.cc:288] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [CANDIDATE]: Term 1 election: Requested vote from peers 25d053f330d747329f158333e7295ac7 (127.8.14.65:45261), a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407)
I20260808 16:59:58.493024 32099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a485f49b846b41ec9aa7f9ea79752345" candidate_uuid: "5c20d9c140184f969e86ad6d3b0bad77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6d4e9cdda2240949054f7005002891c"
I20260808 16:59:58.493052 31911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a485f49b846b41ec9aa7f9ea79752345" candidate_uuid: "5c20d9c140184f969e86ad6d3b0bad77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25d053f330d747329f158333e7295ac7"
I20260808 16:59:58.493113 32099 raft_consensus.cc:3037] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:58.493141 31911 raft_consensus.cc:3037] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 16:59:58.493956 31911 raft_consensus.cc:2454] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c20d9c140184f969e86ad6d3b0bad77 in term 1.
I20260808 16:59:58.494037 32099 raft_consensus.cc:2454] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c20d9c140184f969e86ad6d3b0bad77 in term 1.
I20260808 16:59:58.494153 32189 leader_election.cc:302] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25d053f330d747329f158333e7295ac7, 5c20d9c140184f969e86ad6d3b0bad77; no voters:
I20260808 16:59:58.494311 32374 raft_consensus.cc:2781] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 16:59:58.494457 32374 raft_consensus.cc:686] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [term 1 LEADER]: Becoming Leader. State: Replica: 5c20d9c140184f969e86ad6d3b0bad77, State: Running, Role: LEADER
I20260808 16:59:58.494552 32374 consensus_queue.cc:237] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } } peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } }
I20260808 16:59:58.495464 31712 catalog_manager.cc:5495] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c20d9c140184f969e86ad6d3b0bad77 (127.8.14.67). New cstate: current_term: 1 leader_uuid: "5c20d9c140184f969e86ad6d3b0bad77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } health_report { overall_health: UNKNOWN } } }
I20260808 16:59:58.514762 32358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:58.512640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35840 (local address 127.8.14.67:45431)
0808 16:59:58.512697 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 16:59:58.512699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 16:59:58.512714 (+ 15us) server_negotiation.cc:407] Connection header received
0808 16:59:58.512743 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 16:59:58.512744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 16:59:58.512760 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 16:59:58.512829 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 16:59:58.512952 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.513392 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.514075 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 16:59:58.514248 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 16:59:58.514295 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 16:59:58.514424 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 16:59:58.514515 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 16:59:58.514633 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 16:59:58.514693 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 16:59:58.514711 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 16:59:58.516549 32099 raft_consensus.cc:1261] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c [term 1 FOLLOWER]: Refusing update from remote peer 5c20d9c140184f969e86ad6d3b0bad77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:59:58.516551 31911 raft_consensus.cc:1261] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Refusing update from remote peer 5c20d9c140184f969e86ad6d3b0bad77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 16:59:58.516906 32378 consensus_queue.cc:1035] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:59:58.517027 32374 consensus_queue.cc:1035] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 16:59:58.519888 32383 mvcc.cc:205] Tried to move back new op lower bound from 7316309600321015808 to 7316309600234307584. Current Snapshot: MvccSnapshot[applied={T|T < 7316309600321015808 or (T in {7316309600321015808})}]
I20260808 16:59:58.519989 32384 mvcc.cc:205] Tried to move back new op lower bound from 7316309600321015808 to 7316309600234307584. Current Snapshot: MvccSnapshot[applied={T|T < 7316309600321015808 or (T in {7316309600321015808})}]
I20260808 16:59:58.522064 31713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:59:58.522166 31713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 16:59:58.522941 31713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: Sending DeleteTablet for 3 replicas of tablet a485f49b846b41ec9aa7f9ea79752345
I20260808 16:59:58.523265 32255 tablet_service.cc:1514] Processing DeleteTablet for tablet a485f49b846b41ec9aa7f9ea79752345 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:59:58 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:35838
I20260808 16:59:58.523306 31883 tablet_service.cc:1514] Processing DeleteTablet for tablet a485f49b846b41ec9aa7f9ea79752345 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:59:58 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:44814
I20260808 16:59:58.523308 32066 tablet_service.cc:1514] Processing DeleteTablet for tablet a485f49b846b41ec9aa7f9ea79752345 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:59:58 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:42702
I20260808 16:59:58.523542 32393 tablet_replica.cc:330] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: stopping tablet replica
I20260808 16:59:58.523576 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 31800
I20260808 16:59:58.523651 32393 raft_consensus.cc:2229] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:59:58.523764 32393 raft_consensus.cc:2258] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:59:58.523918 32392 tablet_replica.cc:330] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77: stopping tablet replica
I20260808 16:59:58.524000 32392 raft_consensus.cc:2229] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [term 1 LEADER]: Raft consensus shutting down.
I20260808 16:59:58.528362 32392 raft_consensus.cc:2258] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:59:58.530746 32392 ts_tablet_manager.cc:1905] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:59:58.533520 32392 ts_tablet_manager.cc:1918] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:59:58.533610 32392 log.cc:1196] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/a485f49b846b41ec9aa7f9ea79752345
I20260808 16:59:58.533993 32392 ts_tablet_manager.cc:1939] T a485f49b846b41ec9aa7f9ea79752345 P 5c20d9c140184f969e86ad6d3b0bad77: Deleting consensus metadata
I20260808 16:59:58.539961 31690 catalog_manager.cc:4841] TS 5c20d9c140184f969e86ad6d3b0bad77 (127.8.14.67:45431): tablet a485f49b846b41ec9aa7f9ea79752345 (table test-table [id=09d523fb05884270b5b81e4e8f0d7a02]) successfully deleted
I20260808 16:59:58.547374 32394 tablet_replica.cc:330] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c: stopping tablet replica
I20260808 16:59:58.547511 32394 raft_consensus.cc:2229] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 16:59:58.547636 32394 raft_consensus.cc:2258] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 16:59:58.548151 32394 ts_tablet_manager.cc:1905] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 16:59:58.550190 32394 ts_tablet_manager.cc:1918] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 16:59:58.550262 32394 log.cc:1196] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/a485f49b846b41ec9aa7f9ea79752345
I20260808 16:59:58.550531 32394 ts_tablet_manager.cc:1939] T a485f49b846b41ec9aa7f9ea79752345 P a6d4e9cdda2240949054f7005002891c: Deleting consensus metadata
I20260808 16:59:58.551020 31688 catalog_manager.cc:4841] TS a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407): tablet a485f49b846b41ec9aa7f9ea79752345 (table test-table [id=09d523fb05884270b5b81e4e8f0d7a02]) successfully deleted
W20260808 16:59:58.552650 31690 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.14.65:45261 (error 108)
I20260808 16:59:58.553161 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 31976
I20260808 16:59:58.553198 32362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 16:59:58.553061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:45261 (local address 127.0.0.1:44830)
0808 16:59:58.553117 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 16:59:58.553162 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.65:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 16:59:58.556438 31690 catalog_manager.cc:4584] TS 25d053f330d747329f158333e7295ac7 (127.8.14.65:45261): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a485f49b846b41ec9aa7f9ea79752345: Network error: Client connection negotiation failed: client connection to 127.8.14.65:45261: connect: Connection refused (error 111)
I20260808 16:59:58.574283 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 32164
I20260808 16:59:58.584323 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 31666
I20260808 17:00:18.593155 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:46633
--webserver_interface=127.8.14.126
--webserver_port=35963
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:46633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:00:18.678951 32396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:18.679145 32396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:18.679168 32396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:18.680300 32396 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 17:00:18.680341 32396 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 17:00:18.680354 32396 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 17:00:18.680366 32396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:00:18.680420 32396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:18.680438 32396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:18.680465 32396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:00:18.680486 32396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:00:18.681905 32396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:46633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:46633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=35963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:18.682148 32396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:18.682377 32396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:18.692137 32411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:18.692132 32413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:18.692137 32410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:18.692245 32396 server_base.cc:1034] running on GCE node
I20260808 17:00:18.692667 32396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:18.692974 32396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:18.694135 32396 hybrid_clock.cc:648] HybridClock initialized: now 1786208418694123 us; error 45 us; skew 500 ppm
Aug 08 17:00:18 dist-test-slave-1nl6 krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208418, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:00:18.701493 32396 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 17:00:18.701925 32396 webserver.cc:460] Webserver started at http://127.8.14.126:35963/ using document root <none> and password file <none>
I20260808 17:00:18.702160 32396 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:18.702246 32396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:18.703677 32396 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:00:18.704358 32423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:18.704530 32396 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:00:18.704645 32396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "5e847c66e5364aed8144db9949cd8fca"
format_stamp: "Formatted at 2026-08-08 16:59:57 on dist-test-slave-1nl6"
server_key: "d71e8633554e6e1edfbd41924581cc00"
server_key_iv: "ddb9bd47d641f8aa1bcc42a9cfc4737b"
server_key_version: "encryptionkey@0"
I20260808 17:00:18.705039 32396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:18.723388 32396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:18.725140 32396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:18.725309 32396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:18.729611 32396 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:46633
I20260808 17:00:18.729646 32494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:46633 every 8 connection(s)
I20260808 17:00:18.730089 32396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:00:18.731626 32495 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:00:18.732262 32495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: Bootstrap starting.
I20260808 17:00:18.740332 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 32396
I20260808 17:00:18.740638 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:45261
--local_ip_for_outbound_sockets=127.8.14.65
--tserver_master_addrs=127.8.14.126:46633
--webserver_port=35417
--webserver_interface=127.8.14.65
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:00:18.741227 32495 log.cc:824] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: Log is configured to *not* fsync() on all Append() calls
I20260808 17:00:18.743783 32495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:00:18.744110 32495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: Bootstrap complete.
I20260808 17:00:18.746407 32495 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } }
I20260808 17:00:18.746626 32495 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e847c66e5364aed8144db9949cd8fca, State: Initialized, Role: FOLLOWER
I20260808 17:00:18.746801 32495 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } }
I20260808 17:00:18.746866 32495 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:00:18.746937 32495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:00:18.746992 32495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 1 FOLLOWER]: Advancing to term 2
I20260808 17:00:18.748004 32495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } }
I20260808 17:00:18.748095 32495 leader_election.cc:302] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e847c66e5364aed8144db9949cd8fca; no voters:
I20260808 17:00:18.748322 32495 leader_election.cc:288] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 17:00:18.748358 32499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 2 FOLLOWER]: Leader election won for term 2
I20260808 17:00:18.748545 32499 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 2 LEADER]: Becoming Leader. State: Replica: 5e847c66e5364aed8144db9949cd8fca, State: Running, Role: LEADER
I20260808 17:00:18.748607 32495 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:00:18.748687 32499 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } }
I20260808 17:00:18.748965 32499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e847c66e5364aed8144db9949cd8fca. Latest consensus state: current_term: 2 leader_uuid: "5e847c66e5364aed8144db9949cd8fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } } }
I20260808 17:00:18.749040 32499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [sys.catalog]: This master's current role is: LEADER
I20260808 17:00:18.748992 32502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5e847c66e5364aed8144db9949cd8fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } } }
I20260808 17:00:18.749249 32502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [sys.catalog]: This master's current role is: LEADER
I20260808 17:00:18.749357 32507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:00:18.750174 32507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:00:18.750284 32507 catalog_manager.cc:1259] Loaded cluster ID: 3635b83ffb8c4c09aa883ac513475ce1
I20260808 17:00:18.750334 32507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:00:18.751500 32507 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:00:18.751685 32507 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: Loaded TSK: 0
I20260808 17:00:18.764544 32507 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: Generated new TSK 1
I20260808 17:00:18.764636 32507 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 17:00:18.825441 32498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:18.825691 32498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:18.825730 32498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:18.826977 32498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:18.827051 32498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:18.827095 32498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 17:00:18.828467 32498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:45261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=35417
--tserver_master_addrs=127.8.14.126:46633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:18.828747 32498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:18.828989 32498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:18.835000 32530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:18.835011 32529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:18.835202 32498 server_base.cc:1034] running on GCE node
W20260808 17:00:18.835000 32532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:18.835613 32498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:18.835875 32498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:18.837051 32498 hybrid_clock.cc:648] HybridClock initialized: now 1786208418837047 us; error 38 us; skew 500 ppm
Aug 08 17:00:18 dist-test-slave-1nl6 krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208418, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:00:18.839766 32498 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 17:00:18.840281 32498 webserver.cc:460] Webserver started at http://127.8.14.65:35417/ using document root <none> and password file <none>
I20260808 17:00:18.840528 32498 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:18.840610 32498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:18.842350 32498 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:00:18.843205 32541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:18.843420 32498 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:00:18.843537 32498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "25d053f330d747329f158333e7295ac7"
format_stamp: "Formatted at 2026-08-08 16:59:57 on dist-test-slave-1nl6"
server_key: "5d209666db4cb765e5e26b5633305c43"
server_key_iv: "0b7d8a4a4bebc4d6fc3562762dab6c72"
server_key_version: "encryptionkey@0"
I20260808 17:00:18.843990 32498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:18.853662 32498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:18.855774 32498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:18.855926 32498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:18.856217 32498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:00:18.856738 32552 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 17:00:18.858175 32498 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 17:00:18.858225 32498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 17:00:18.858259 32498 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 17:00:18.858949 32498 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 17:00:18.858994 32498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260808 17:00:18.859020 32552 tablet_bootstrap.cc:492] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: Bootstrap starting.
I20260808 17:00:18.868251 32498 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:45261
I20260808 17:00:18.868530 32696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:45261 every 8 connection(s)
I20260808 17:00:18.868786 32498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 17:00:18.870188 32552 log.cc:824] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: Log is configured to *not* fsync() on all Append() calls
I20260808 17:00:18.872742 32552 tablet_bootstrap.cc:492] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:00:18.873044 32552 tablet_bootstrap.cc:492] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: Bootstrap complete.
I20260808 17:00:18.874264 32552 ts_tablet_manager.cc:1397] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.000s
I20260808 17:00:18.875562 32552 raft_consensus.cc:348] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } } peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } }
I20260808 17:00:18.875871 32552 raft_consensus.cc:729] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25d053f330d747329f158333e7295ac7, State: Initialized, Role: FOLLOWER
I20260808 17:00:18.876011 32552 consensus_queue.cc:260] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } } peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } }
I20260808 17:00:18.876257 32552 ts_tablet_manager.cc:1428] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: Time spent starting tablet: real 0.002s user 0.001s sys 0.004s
Aug 08 17:00:18 dist-test-slave-1nl6 krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208418, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:00:18.876920 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 32498
I20260808 17:00:18.877246 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:34407
--local_ip_for_outbound_sockets=127.8.14.66
--tserver_master_addrs=127.8.14.126:46633
--webserver_port=33525
--webserver_interface=127.8.14.66
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:00:18.881358 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:18.869715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:47463)
0808 17:00:18.870453 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:18.870495 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:18.871043 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:18.871482 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:18.871486 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:18.871764 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:00:18.871935 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:18.871940 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:18.873243 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:18.873245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:18.874064 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:18.874067 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:18.874770 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:18.874772 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:18.874961 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:18.875536 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:00:18.875544 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:00:18.877178 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:00:18.879499 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:18.879505 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:18.879508 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:18.879805 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:18.880106 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:18.880107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:18.880108 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:18.880260 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:18.880754 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:00:18.880762 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:00:18.880902 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 17:00:18.881001 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 17:00:18.881163 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":93,"threads_started":1}
I20260808 17:00:18.881865 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:18.869787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:47463 (local address 127.8.14.126:46633)
0808 17:00:18.870397 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:18.870407 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:18.870616 (+ 209us) server_negotiation.cc:407] Connection header received
0808 17:00:18.871273 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:18.871278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:18.871320 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:18.871451 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:18.872053 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:18.873136 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:18.874180 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:18.874656 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:18.874780 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:18.877303 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:18.877308 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:18.877309 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:00:18.877323 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:18.879397 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:18.879904 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:18.879908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:18.879908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:18.880022 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:18.880349 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:18.880351 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:18.880351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:18.880643 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:18.880722 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:18.881529 (+ 807us) server_negotiation.cc:299] Negotiation successful
0808 17:00:18.881666 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":86,"threads_started":1}
I20260808 17:00:18.882495 32697 heartbeater.cc:347] Connected to a master server at 127.8.14.126:46633
I20260808 17:00:18.882588 32697 heartbeater.cc:464] Registering TS with master...
I20260808 17:00:18.882762 32697 heartbeater.cc:511] Master 127.8.14.126:46633 requested a full tablet report, sending...
I20260808 17:00:18.883677 32451 ts_manager.cc:194] Registered new tserver with Master: 25d053f330d747329f158333e7295ac7 (127.8.14.65:45261)
I20260808 17:00:18.884886 32451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:47463
I20260808 17:00:18.885807 32697 heartbeater.cc:503] Master 127.8.14.126:46633 was elected leader, sending a full tablet report...
I20260808 17:00:18.891377 32708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:18.884611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48390 (local address 127.8.14.65:45261)
0808 17:00:18.887260 (+ 2649us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:18.887264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:18.887272 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:00:18.887301 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:18.887302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:18.887315 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:18.887384 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:00:18.887584 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:18.888671 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:18.890255 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:18.890991 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:18.891065 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:18.891115 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:18.891284 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 17:00:18.891320 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2621,"thread_start_us":83,"threads_started":1}
I20260808 17:00:18.891769 32707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:18.884364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:45261 (local address 127.0.0.1:48390)
0808 17:00:18.887035 (+ 2671us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:18.887085 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:18.887148 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:18.887420 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:18.887423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:18.887426 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:00:18.887496 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:18.887499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:18.888763 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:18.888764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:18.890137 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:18.890141 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:18.891091 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:18.891092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:18.891174 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:18.891189 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 17:00:18.891259 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 17:00:18.891307 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2641,"thread_start_us":85,"threads_started":1}
I20260808 17:00:18.892185 32614 tablet_service.cc:1514] Processing DeleteTablet for tablet a485f49b846b41ec9aa7f9ea79752345 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 16:59:58 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:48390
I20260808 17:00:18.892732 32710 tablet_replica.cc:330] stopping tablet replica
I20260808 17:00:18.892838 32710 raft_consensus.cc:2229] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:00:18.892902 32710 raft_consensus.cc:2258] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:00:18.893224 32710 ts_tablet_manager.cc:1905] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:00:18.895337 32710 ts_tablet_manager.cc:1918] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:00:18.895416 32710 log.cc:1196] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal/wals/a485f49b846b41ec9aa7f9ea79752345
I20260808 17:00:18.895663 32710 ts_tablet_manager.cc:1939] T a485f49b846b41ec9aa7f9ea79752345 P 25d053f330d747329f158333e7295ac7: Deleting consensus metadata
I20260808 17:00:18.896231 32426 catalog_manager.cc:4841] TS 25d053f330d747329f158333e7295ac7 (127.8.14.65:45261): tablet a485f49b846b41ec9aa7f9ea79752345 (table test-table [id=09d523fb05884270b5b81e4e8f0d7a02]) successfully deleted
W20260808 17:00:18.963354 32704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:18.963536 32704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:18.963558 32704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:18.964623 32704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:18.964658 32704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:18.964684 32704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 17:00:18.966032 32704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:34407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=33525
--tserver_master_addrs=127.8.14.126:46633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:18.966235 32704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:18.966390 32704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:18.971454 32720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:18.971459 32717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:18.971534 32704 server_base.cc:1034] running on GCE node
W20260808 17:00:18.971459 32718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:18.971832 32704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:18.972069 32704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:18.973243 32704 hybrid_clock.cc:648] HybridClock initialized: now 1786208418973248 us; error 46 us; skew 500 ppm
Aug 08 17:00:18 dist-test-slave-1nl6 krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208418, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:00:18.975672 32704 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 17:00:18.976083 32704 webserver.cc:460] Webserver started at http://127.8.14.66:33525/ using document root <none> and password file <none>
I20260808 17:00:18.976321 32704 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:18.976409 32704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:18.977749 32704 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:00:18.978300 32732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:18.978458 32704 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:00:18.978575 32704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "a6d4e9cdda2240949054f7005002891c"
format_stamp: "Formatted at 2026-08-08 16:59:58 on dist-test-slave-1nl6"
server_key: "98d2ef0974494be6d4df7656980c2920"
server_key_iv: "dba150f6d4f7d2544d79cb9162b46d17"
server_key_version: "encryptionkey@0"
I20260808 17:00:18.978883 32704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:18.991767 32704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:18.993723 32704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:18.993906 32704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:18.994174 32704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:00:18.994506 32704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:00:18.994565 32704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:18.994601 32704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:00:18.994648 32704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:19.000658 32704 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:34407
I20260808 17:00:19.000708 415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:34407 every 8 connection(s)
I20260808 17:00:19.001050 32704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 17:00:19.003026 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 32704
I20260808 17:00:19.003331 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:45431
--local_ip_for_outbound_sockets=127.8.14.67
--tserver_master_addrs=127.8.14.126:46633
--webserver_port=43221
--webserver_interface=127.8.14.67
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 17:00:19 dist-test-slave-1nl6 krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208418, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:00:19.012176 418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:19.001758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:40005)
0808 17:00:19.001967 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:19.002004 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:19.002521 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:19.002775 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:19.002779 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:19.003549 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:00:19.003791 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:19.003796 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.004511 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.004512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:19.005718 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:19.005721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.006130 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.006132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:19.006301 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:19.006790 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:00:19.006800 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:00:19.008667 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:00:19.010717 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:19.010724 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:19.010726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:19.011033 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:19.011255 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:19.011256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:19.011256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:19.011366 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:19.011677 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:00:19.011681 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:00:19.011801 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 17:00:19.011875 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 17:00:19.012004 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":113,"threads_started":1}
I20260808 17:00:19.012205 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:19.001803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:40005 (local address 127.8.14.126:46633)
0808 17:00:19.001868 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:19.001870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:19.002088 (+ 218us) server_negotiation.cc:407] Connection header received
0808 17:00:19.002630 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:19.002632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:19.002647 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:19.002705 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:19.003902 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.004410 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.005821 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.006037 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.006100 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:19.008806 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:19.008808 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:19.008808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:00:19.008817 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:19.010621 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:19.011111 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:19.011113 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:19.011113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:19.011182 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:19.011449 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:19.011450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:19.011450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:19.011607 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:19.011684 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:19.011948 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 17:00:19.012054 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 17:00:19.012691 416 heartbeater.cc:347] Connected to a master server at 127.8.14.126:46633
I20260808 17:00:19.012811 416 heartbeater.cc:464] Registering TS with master...
I20260808 17:00:19.012993 416 heartbeater.cc:511] Master 127.8.14.126:46633 requested a full tablet report, sending...
I20260808 17:00:19.013413 32451 ts_manager.cc:194] Registered new tserver with Master: a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407)
I20260808 17:00:19.013967 32451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:40005
W20260808 17:00:19.088243 419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:19.088428 419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:19.088449 419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:19.089617 419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:19.089656 419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:19.089681 419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 17:00:19.091245 419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:45431
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=43221
--tserver_master_addrs=127.8.14.126:46633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:19.091488 419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:19.091728 419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:19.102191 430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:19.102232 426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:19.102192 424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:19.102422 419 server_base.cc:1034] running on GCE node
I20260808 17:00:19.102660 419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:19.102949 419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:19.104106 419 hybrid_clock.cc:648] HybridClock initialized: now 1786208419104095 us; error 27 us; skew 500 ppm
Aug 08 17:00:19 dist-test-slave-1nl6 krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208419, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:00:19.106415 419 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 17:00:19.106819 419 webserver.cc:460] Webserver started at http://127.8.14.67:43221/ using document root <none> and password file <none>
I20260808 17:00:19.107064 419 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:19.107147 419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:19.108503 419 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:00:19.109117 439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:19.109282 419 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:19.109377 419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "5c20d9c140184f969e86ad6d3b0bad77"
format_stamp: "Formatted at 2026-08-08 16:59:58 on dist-test-slave-1nl6"
server_key: "6dc565364b867e034a4db7585bc26e74"
server_key_iv: "9ae6da08c25e0ffaefd6edcfa5fba1b8"
server_key_version: "encryptionkey@0"
I20260808 17:00:19.109683 419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:19.127638 419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:19.129973 419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:19.130086 419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:19.130272 419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:00:19.130528 419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:00:19.130559 419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:19.130580 419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:00:19.130594 419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:19.136281 419 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:45431
I20260808 17:00:19.136361 599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:45431 every 8 connection(s)
I20260808 17:00:19.136688 419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 17:00:19.139658 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 419
Aug 08 17:00:19 dist-test-slave-1nl6 krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208419, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:00:19.146620 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:19.137466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:49449 (local address 127.8.14.126:46633)
0808 17:00:19.137513 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:19.137515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:19.137708 (+ 193us) server_negotiation.cc:407] Connection header received
0808 17:00:19.138222 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:19.138224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:19.138238 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:19.138293 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:19.139314 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.140052 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.141502 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.141704 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.141782 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:19.144028 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:19.144029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:19.144030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:00:19.144038 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:19.145308 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:19.145685 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:19.145686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:19.145687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:19.145744 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:19.145992 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:19.145993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:19.145994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:19.146100 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:19.146170 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:19.146443 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 17:00:19.146533 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:00:19.146811 602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:19.137442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:49449)
0808 17:00:19.137588 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:19.137622 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:19.138115 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:19.138368 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:19.138372 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:19.139011 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:00:19.139204 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:19.139209 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.140156 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.140158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:19.141410 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:19.141413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.141782 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.141783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:19.141906 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:19.142394 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:00:19.142402 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:00:19.143931 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:00:19.145389 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:19.145395 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:19.145397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:19.145609 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:19.145814 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:19.145815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:19.145815 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:19.145925 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:19.146174 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:00:19.146180 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:00:19.146310 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 17:00:19.146407 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 17:00:19.146551 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":65,"threads_started":1}
I20260808 17:00:19.146919 604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:19.140080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33654 (local address 127.8.14.126:46633)
0808 17:00:19.140197 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:19.140199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:19.140289 (+ 90us) server_negotiation.cc:407] Connection header received
0808 17:00:19.140356 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:19.140356 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:19.140366 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:19.140423 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:19.141176 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.141617 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.142577 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.142786 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.142874 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:19.144186 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:19.144188 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:19.144188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:00:19.144196 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:19.145716 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:19.146071 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:19.146072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:19.146072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:19.146128 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:19.146331 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:19.146332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:19.146332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:19.146442 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:19.146621 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:19.146660 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 17:00:19.146764 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":53,"threads_started":1}
I20260808 17:00:19.147246 600 heartbeater.cc:347] Connected to a master server at 127.8.14.126:46633
I20260808 17:00:19.147344 600 heartbeater.cc:464] Registering TS with master...
I20260808 17:00:19.147545 600 heartbeater.cc:511] Master 127.8.14.126:46633 requested a full tablet report, sending...
I20260808 17:00:19.147934 32450 ts_manager.cc:194] Registered new tserver with Master: 5c20d9c140184f969e86ad6d3b0bad77 (127.8.14.67:45431)
I20260808 17:00:19.148339 32450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:49449
I20260808 17:00:19.158782 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:00:19.161314 604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:19.159217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33660 (local address 127.8.14.126:46633)
0808 17:00:19.159266 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:19.159268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:19.159354 (+ 86us) server_negotiation.cc:407] Connection header received
0808 17:00:19.159400 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:19.159400 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:19.159429 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:19.159489 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:00:19.159573 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.159992 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.160673 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.160841 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.160886 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:19.161056 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:00:19.161097 (+ 41us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 17:00:19.161271 (+ 174us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33660: token expired
Metrics: {"server-negotiator.queue_time_us":31}
W20260808 17:00:19.161374 604 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33660: token expired
W20260808 17:00:19.161509 32349 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.14.126:46633)
I20260808 17:00:19.167183 604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:19.161773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33662 (local address 127.8.14.126:46633)
0808 17:00:19.161833 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:19.161835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:19.161856 (+ 21us) server_negotiation.cc:407] Connection header received
0808 17:00:19.161873 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:19.161873 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:19.161880 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:19.161918 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:19.162384 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.162785 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.163511 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.163672 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.163720 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:19.164759 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:19.164760 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:19.164760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:00:19.164766 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:19.166100 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:19.166449 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:19.166449 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:19.166449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:19.166496 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:19.166665 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:19.166665 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:19.166665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:19.166784 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:19.166854 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:19.167021 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 17:00:19.167079 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 17:00:19.168288 32450 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:00:19.169842 32450 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:00:19.172243 32614 tablet_service.cc:1467] Processing CreateTablet for tablet 67446e3a4a0a4edfab0ae5a4b6eb3938 (DEFAULT_TABLE table=test-table [id=73807664c3ef4e2c85b3f493e6a7e0a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:00:19.172456 32614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67446e3a4a0a4edfab0ae5a4b6eb3938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:00:19.173902 32552 tablet_bootstrap.cc:492] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7: Bootstrap starting.
I20260808 17:00:19.174238 32552 tablet_bootstrap.cc:654] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7: Neither blocks nor log segments found. Creating new log.
I20260808 17:00:19.174860 32552 tablet_bootstrap.cc:492] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7: No bootstrap required, opened a new log
I20260808 17:00:19.174938 32552 ts_tablet_manager.cc:1397] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 17:00:19.175058 32552 raft_consensus.cc:348] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } } peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } }
I20260808 17:00:19.175132 32552 raft_consensus.cc:374] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:00:19.175185 32552 raft_consensus.cc:729] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25d053f330d747329f158333e7295ac7, State: Initialized, Role: FOLLOWER
I20260808 17:00:19.175240 32552 consensus_queue.cc:260] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } } peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } }
I20260808 17:00:19.175346 32552 ts_tablet_manager.cc:1428] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:19.175745 32707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:19.171651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:34407 (local address 127.0.0.1:45496)
0808 17:00:19.171705 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:19.171718 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:19.171760 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:19.172174 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:19.172175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:19.172177 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:00:19.172211 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:19.172213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.173398 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.173399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:19.174211 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:19.174214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.175521 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.175522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:19.175581 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:19.175584 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:00:19.175646 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 17:00:19.175682 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 17:00:19.175921 608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:19.171767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45496 (local address 127.8.14.66:34407)
0808 17:00:19.171969 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:19.171973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:19.171983 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:00:19.172027 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:19.172029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:19.172059 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:19.172159 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:00:19.172285 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.173306 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.174370 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.175450 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.175526 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:19.175592 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:19.175660 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 17:00:19.175705 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":129,"threads_started":1}
I20260808 17:00:19.176640 334 tablet_service.cc:1467] Processing CreateTablet for tablet 67446e3a4a0a4edfab0ae5a4b6eb3938 (DEFAULT_TABLE table=test-table [id=73807664c3ef4e2c85b3f493e6a7e0a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:00:19.177000 334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67446e3a4a0a4edfab0ae5a4b6eb3938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:00:19.177459 606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:19.171700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:45431 (local address 127.0.0.1:36278)
0808 17:00:19.171864 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:19.171875 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:19.171923 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:19.174173 (+ 2250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:19.174174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:19.174176 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:00:19.174223 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:19.174224 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.175313 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.175314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:19.176427 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:19.176430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.177291 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.177291 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:19.177338 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:19.177341 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:00:19.177384 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 17:00:19.177413 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":117,"threads_started":1}
I20260808 17:00:19.177531 609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:19.172075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36278 (local address 127.8.14.67:45431)
0808 17:00:19.174030 (+ 1955us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:19.174035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:19.174045 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:00:19.174082 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:19.174084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:19.174099 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:19.174179 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:00:19.174285 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.175238 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.176525 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.177204 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.177281 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:19.177346 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:19.177410 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 17:00:19.177455 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1919,"thread_start_us":113,"threads_started":1}
I20260808 17:00:19.178220 508 tablet_service.cc:1467] Processing CreateTablet for tablet 67446e3a4a0a4edfab0ae5a4b6eb3938 (DEFAULT_TABLE table=test-table [id=73807664c3ef4e2c85b3f493e6a7e0a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:00:19.178572 508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67446e3a4a0a4edfab0ae5a4b6eb3938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:00:19.180581 611 tablet_bootstrap.cc:492] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c: Bootstrap starting.
I20260808 17:00:19.181151 611 tablet_bootstrap.cc:654] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c: Neither blocks nor log segments found. Creating new log.
I20260808 17:00:19.181782 611 log.cc:824] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c: Log is configured to *not* fsync() on all Append() calls
I20260808 17:00:19.181880 614 tablet_bootstrap.cc:492] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77: Bootstrap starting.
I20260808 17:00:19.182518 614 tablet_bootstrap.cc:654] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77: Neither blocks nor log segments found. Creating new log.
I20260808 17:00:19.182548 611 tablet_bootstrap.cc:492] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c: No bootstrap required, opened a new log
I20260808 17:00:19.182642 611 ts_tablet_manager.cc:1397] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:00:19.183109 614 log.cc:824] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77: Log is configured to *not* fsync() on all Append() calls
I20260808 17:00:19.183836 614 tablet_bootstrap.cc:492] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77: No bootstrap required, opened a new log
I20260808 17:00:19.183918 614 ts_tablet_manager.cc:1397] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:00:19.184355 611 raft_consensus.cc:348] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } } peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } }
I20260808 17:00:19.184497 611 raft_consensus.cc:374] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:00:19.184518 611 raft_consensus.cc:729] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6d4e9cdda2240949054f7005002891c, State: Initialized, Role: FOLLOWER
I20260808 17:00:19.184648 611 consensus_queue.cc:260] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } } peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } }
I20260808 17:00:19.184930 611 ts_tablet_manager.cc:1428] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:00:19.185065 416 heartbeater.cc:503] Master 127.8.14.126:46633 was elected leader, sending a full tablet report...
I20260808 17:00:19.185701 614 raft_consensus.cc:348] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } } peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } }
I20260808 17:00:19.185865 614 raft_consensus.cc:374] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:00:19.185911 614 raft_consensus.cc:729] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c20d9c140184f969e86ad6d3b0bad77, State: Initialized, Role: FOLLOWER
I20260808 17:00:19.186038 614 consensus_queue.cc:260] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } } peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } }
I20260808 17:00:19.186299 614 ts_tablet_manager.cc:1428] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:00:19.186367 600 heartbeater.cc:503] Master 127.8.14.126:46633 was elected leader, sending a full tablet report...
W20260808 17:00:19.251569 417 tablet.cc:2369] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:00:19.369858 32698 tablet.cc:2369] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:00:19.374018 32703 raft_consensus.cc:482] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:00:19.374172 32703 raft_consensus.cc:504] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } } peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } }
I20260808 17:00:19.374558 32703 leader_election.cc:288] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407), 5c20d9c140184f969e86ad6d3b0bad77 (127.8.14.67:45431)
I20260808 17:00:19.377691 32699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:19.374753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:45431 (local address 127.8.14.65:47301)
0808 17:00:19.374847 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:19.374860 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:19.374926 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:19.375137 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:19.375138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:19.375139 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:00:19.375227 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:19.375229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.376042 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.376043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:19.376815 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:19.376818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.377388 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.377389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:19.377455 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:19.377482 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 17:00:19.377536 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 17:00:19.377587 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 17:00:19.377794 609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:19.374864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:47301 (local address 127.8.14.67:45431)
0808 17:00:19.374967 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:19.374969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:19.374978 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:00:19.375015 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:19.375016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:19.375026 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:19.375119 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:00:19.375306 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.375947 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.376911 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.377303 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.377631 (+ 328us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:19.377647 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:19.377669 (+ 22us) server_negotiation.cc:299] Negotiation successful
0808 17:00:19.377715 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 17:00:19.378211 536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67446e3a4a0a4edfab0ae5a4b6eb3938" candidate_uuid: "25d053f330d747329f158333e7295ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c20d9c140184f969e86ad6d3b0bad77" is_pre_election: true
I20260808 17:00:19.378388 536 raft_consensus.cc:2454] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25d053f330d747329f158333e7295ac7 in term 0.
I20260808 17:00:19.378417 608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:19.374842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:36235 (local address 127.8.14.66:34407)
0808 17:00:19.374912 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:19.374914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:19.375119 (+ 205us) server_negotiation.cc:407] Connection header received
0808 17:00:19.375151 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:19.375152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:19.375164 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:19.375215 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:00:19.375689 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.376181 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.377415 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.378001 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.378085 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:19.378096 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:19.378308 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 17:00:19.378349 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:00:19.378432 620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:19.374756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:34407 (local address 127.8.14.65:36235)
0808 17:00:19.375003 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:19.375012 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:19.375068 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:19.375241 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:19.375242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:19.375244 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:00:19.375605 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:19.375606 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.376261 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.376262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:19.377313 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:19.377316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.378134 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.378135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:19.378219 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:19.378224 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:00:19.378278 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 17:00:19.378333 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":78,"thread_start_us":102,"threads_started":1}
I20260808 17:00:19.378616 32546 leader_election.cc:302] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25d053f330d747329f158333e7295ac7, 5c20d9c140184f969e86ad6d3b0bad77; no voters:
I20260808 17:00:19.378743 32703 raft_consensus.cc:2781] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:00:19.378813 32703 raft_consensus.cc:482] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:00:19.378849 32703 raft_consensus.cc:3037] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:00:19.378924 355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67446e3a4a0a4edfab0ae5a4b6eb3938" candidate_uuid: "25d053f330d747329f158333e7295ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6d4e9cdda2240949054f7005002891c" is_pre_election: true
I20260808 17:00:19.379061 355 raft_consensus.cc:2454] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25d053f330d747329f158333e7295ac7 in term 0.
I20260808 17:00:19.379757 32703 raft_consensus.cc:504] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } } peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } }
I20260808 17:00:19.379973 32703 leader_election.cc:288] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 1 election: Requested vote from peers a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407), 5c20d9c140184f969e86ad6d3b0bad77 (127.8.14.67:45431)
I20260808 17:00:19.380270 355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67446e3a4a0a4edfab0ae5a4b6eb3938" candidate_uuid: "25d053f330d747329f158333e7295ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6d4e9cdda2240949054f7005002891c"
I20260808 17:00:19.380293 536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67446e3a4a0a4edfab0ae5a4b6eb3938" candidate_uuid: "25d053f330d747329f158333e7295ac7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c20d9c140184f969e86ad6d3b0bad77"
I20260808 17:00:19.380358 355 raft_consensus.cc:3037] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:00:19.380365 536 raft_consensus.cc:3037] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:00:19.381151 355 raft_consensus.cc:2454] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25d053f330d747329f158333e7295ac7 in term 1.
I20260808 17:00:19.381292 536 raft_consensus.cc:2454] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25d053f330d747329f158333e7295ac7 in term 1.
I20260808 17:00:19.381487 32545 leader_election.cc:302] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25d053f330d747329f158333e7295ac7, a6d4e9cdda2240949054f7005002891c; no voters:
I20260808 17:00:19.382604 32703 raft_consensus.cc:2781] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:00:19.382754 32703 raft_consensus.cc:686] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 LEADER]: Becoming Leader. State: Replica: 25d053f330d747329f158333e7295ac7, State: Running, Role: LEADER
I20260808 17:00:19.382828 32703 consensus_queue.cc:237] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } } peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } }
I20260808 17:00:19.383978 32450 catalog_manager.cc:5495] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25d053f330d747329f158333e7295ac7 (127.8.14.65). New cstate: current_term: 1 leader_uuid: "25d053f330d747329f158333e7295ac7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } health_report { overall_health: HEALTHY } } }
W20260808 17:00:19.388756 601 tablet.cc:2369] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:00:19.401965 626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:19.399789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48400 (local address 127.8.14.65:45261)
0808 17:00:19.399896 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:19.399897 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:19.399905 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:00:19.399936 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:19.399936 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:19.399954 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:19.400018 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:00:19.400119 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.400586 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.401278 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:19.401469 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:19.401526 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:19.401646 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:00:19.401742 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:00:19.401798 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:19.401833 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 17:00:19.401850 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":58,"threads_started":1}
I20260808 17:00:19.403314 536 raft_consensus.cc:1261] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77 [term 1 FOLLOWER]: Refusing update from remote peer 25d053f330d747329f158333e7295ac7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:00:19.403328 355 raft_consensus.cc:1261] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c [term 1 FOLLOWER]: Refusing update from remote peer 25d053f330d747329f158333e7295ac7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:00:19.403612 621 consensus_queue.cc:1035] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:00:19.403765 32703 consensus_queue.cc:1035] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:00:19.405984 628 mvcc.cc:205] Tried to move back new op lower bound from 7316309685873696768 to 7316309685792923648. Current Snapshot: MvccSnapshot[applied={T|T < 7316309685873696768}]
I20260808 17:00:19.407733 630 mvcc.cc:205] Tried to move back new op lower bound from 7316309685873696768 to 7316309685792923648. Current Snapshot: MvccSnapshot[applied={T|T < 7316309685873696768 or (T in {7316309685873696768})}]
I20260808 17:00:19.408164 32450 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:00:19.408282 32450 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:00:19.409008 32450 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: Sending DeleteTablet for 3 replicas of tablet 67446e3a4a0a4edfab0ae5a4b6eb3938
I20260808 17:00:19.409313 334 tablet_service.cc:1514] Processing DeleteTablet for tablet 67446e3a4a0a4edfab0ae5a4b6eb3938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:00:19 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:45496
I20260808 17:00:19.409390 32614 tablet_service.cc:1514] Processing DeleteTablet for tablet 67446e3a4a0a4edfab0ae5a4b6eb3938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:00:19 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:48390
I20260808 17:00:19.409402 508 tablet_service.cc:1514] Processing DeleteTablet for tablet 67446e3a4a0a4edfab0ae5a4b6eb3938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:00:19 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:36278
I20260808 17:00:19.409631 639 tablet_replica.cc:330] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c: stopping tablet replica
I20260808 17:00:19.409682 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 32498
I20260808 17:00:19.409734 639 raft_consensus.cc:2229] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:00:19.414997 639 raft_consensus.cc:2258] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:00:19.416069 640 tablet_replica.cc:330] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77: stopping tablet replica
I20260808 17:00:19.416191 640 raft_consensus.cc:2229] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:00:19.416189 639 ts_tablet_manager.cc:1905] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:00:19.416299 640 raft_consensus.cc:2258] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:00:19.416746 640 ts_tablet_manager.cc:1905] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:00:19.418404 640 ts_tablet_manager.cc:1918] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:00:19.418503 640 log.cc:1196] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/67446e3a4a0a4edfab0ae5a4b6eb3938
I20260808 17:00:19.418520 639 ts_tablet_manager.cc:1918] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:00:19.418598 639 log.cc:1196] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/67446e3a4a0a4edfab0ae5a4b6eb3938
I20260808 17:00:19.418808 640 ts_tablet_manager.cc:1939] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 5c20d9c140184f969e86ad6d3b0bad77: Deleting consensus metadata
I20260808 17:00:19.418934 639 ts_tablet_manager.cc:1939] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P a6d4e9cdda2240949054f7005002891c: Deleting consensus metadata
I20260808 17:00:19.419288 32426 catalog_manager.cc:4841] TS 5c20d9c140184f969e86ad6d3b0bad77 (127.8.14.67:45431): tablet 67446e3a4a0a4edfab0ae5a4b6eb3938 (table test-table [id=73807664c3ef4e2c85b3f493e6a7e0a4]) successfully deleted
I20260808 17:00:19.419330 32425 catalog_manager.cc:4841] TS a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407): tablet 67446e3a4a0a4edfab0ae5a4b6eb3938 (table test-table [id=73807664c3ef4e2c85b3f493e6a7e0a4]) successfully deleted
W20260808 17:00:19.422739 32426 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.14.65:45261 (error 108)
I20260808 17:00:19.423010 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 32704
I20260808 17:00:19.424718 606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:19.423143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:45261 (local address 127.0.0.1:48406)
0808 17:00:19.424612 (+ 1469us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:19.424673 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.65:45261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1443}
W20260808 17:00:19.429965 32426 catalog_manager.cc:4584] TS 25d053f330d747329f158333e7295ac7 (127.8.14.65:45261): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 67446e3a4a0a4edfab0ae5a4b6eb3938: Network error: Client connection negotiation failed: client connection to 127.8.14.65:45261: connect: Connection refused (error 111)
I20260808 17:00:19.434721 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 419
I20260808 17:00:19.443573 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 32396
I20260808 17:00:39.453401 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:46633
--webserver_interface=127.8.14.126
--webserver_port=35963
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:46633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:00:39.537320 645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:39.537554 645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:39.537591 645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:39.538743 645 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 17:00:39.538812 645 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 17:00:39.538836 645 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 17:00:39.538856 645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:00:39.538885 645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:39.538932 645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:39.538959 645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:00:39.538990 645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:00:39.540458 645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:46633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:46633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=35963
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:39.540746 645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:39.540968 645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:39.546060 653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:39.546041 658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:39.546072 645 server_base.cc:1034] running on GCE node
W20260808 17:00:39.546037 656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:39.546597 645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:39.546870 645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:39.548084 645 hybrid_clock.cc:648] HybridClock initialized: now 1786208439548071 us; error 28 us; skew 500 ppm
Aug 08 17:00:39 dist-test-slave-1nl6 krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208439, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:00:39.556267 645 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 17:00:39.556787 645 webserver.cc:460] Webserver started at http://127.8.14.126:35963/ using document root <none> and password file <none>
I20260808 17:00:39.557054 645 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:39.557108 645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:39.558633 645 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 17:00:39.559499 672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:39.559708 645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:00:39.559823 645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "5e847c66e5364aed8144db9949cd8fca"
format_stamp: "Formatted at 2026-08-08 16:59:57 on dist-test-slave-1nl6"
server_key: "d71e8633554e6e1edfbd41924581cc00"
server_key_iv: "ddb9bd47d641f8aa1bcc42a9cfc4737b"
server_key_version: "encryptionkey@0"
I20260808 17:00:39.560253 645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:39.571190 645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:39.573153 645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:39.573312 645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:39.577831 645 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:46633
I20260808 17:00:39.577865 739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:46633 every 8 connection(s)
I20260808 17:00:39.578259 645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:00:39.579190 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 645
I20260808 17:00:39.579471 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:45261
--local_ip_for_outbound_sockets=127.8.14.65
--tserver_master_addrs=127.8.14.126:46633
--webserver_port=35417
--webserver_interface=127.8.14.65
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:00:39.579926 740 sys_catalog.cc:263] Verifying existing consensus state
I20260808 17:00:39.580586 740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: Bootstrap starting.
I20260808 17:00:39.589003 740 log.cc:824] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: Log is configured to *not* fsync() on all Append() calls
I20260808 17:00:39.591411 740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:00:39.591725 740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: Bootstrap complete.
I20260808 17:00:39.593981 740 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } }
I20260808 17:00:39.594200 740 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e847c66e5364aed8144db9949cd8fca, State: Initialized, Role: FOLLOWER
I20260808 17:00:39.594376 740 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } }
I20260808 17:00:39.594441 740 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:00:39.594506 740 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:00:39.594560 740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 2 FOLLOWER]: Advancing to term 3
I20260808 17:00:39.595634 740 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } }
I20260808 17:00:39.595726 740 leader_election.cc:302] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e847c66e5364aed8144db9949cd8fca; no voters:
I20260808 17:00:39.595980 740 leader_election.cc:288] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 17:00:39.596019 748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 3 FOLLOWER]: Leader election won for term 3
I20260808 17:00:39.596154 748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [term 3 LEADER]: Becoming Leader. State: Replica: 5e847c66e5364aed8144db9949cd8fca, State: Running, Role: LEADER
I20260808 17:00:39.596261 740 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:00:39.596242 748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } }
I20260808 17:00:39.596580 749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5e847c66e5364aed8144db9949cd8fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } } }
I20260808 17:00:39.596577 748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e847c66e5364aed8144db9949cd8fca. Latest consensus state: current_term: 3 leader_uuid: "5e847c66e5364aed8144db9949cd8fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e847c66e5364aed8144db9949cd8fca" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46633 } } }
I20260808 17:00:39.596699 749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [sys.catalog]: This master's current role is: LEADER
I20260808 17:00:39.596715 748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca [sys.catalog]: This master's current role is: LEADER
I20260808 17:00:39.598270 769 catalog_manager.cc:1259] Loaded cluster ID: 3635b83ffb8c4c09aa883ac513475ce1
I20260808 17:00:39.598332 769 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: loading cluster ID for follower catalog manager: success
I20260808 17:00:39.599491 769 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5e847c66e5364aed8144db9949cd8fca: acquiring CA information for follower catalog manager: success
F20260808 17:00:39.599864 769 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f170c3eb9ad google::LogMessage::Fail() at ??:0
@ 0x7f170c3ee88a google::LogMessage::SendToLog() at ??:0
@ 0x7f170c3eb4de google::LogMessage::Flush() at ??:0
@ 0x7f170c3eca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f170eb58104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f17143446b9 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f1714344989 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f1714344e4e kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f17143478c2 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f170e2ae954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f170f1126db start_thread at ??:0
@ 0x7f170aee961f clone at ??:0
W20260808 17:00:39.679749 745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:39.679934 745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:39.679956 745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:39.681154 745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:39.681195 745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:39.681222 745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 17:00:39.682677 745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:45261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=35417
--tserver_master_addrs=127.8.14.126:46633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:39.682906 745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:39.683202 745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:39.689239 798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:39.689527 796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:39.689476 745 server_base.cc:1034] running on GCE node
W20260808 17:00:39.689249 795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:39.689841 745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:39.690156 745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:39.691344 745 hybrid_clock.cc:648] HybridClock initialized: now 1786208439691330 us; error 28 us; skew 500 ppm
Aug 08 17:00:39 dist-test-slave-1nl6 krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208439, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:00:39.694356 745 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 17:00:39.694800 745 webserver.cc:460] Webserver started at http://127.8.14.65:35417/ using document root <none> and password file <none>
I20260808 17:00:39.695123 745 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:39.695212 745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:39.696558 745 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:00:39.697314 809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:39.697491 745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:00:39.697598 745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "25d053f330d747329f158333e7295ac7"
format_stamp: "Formatted at 2026-08-08 16:59:57 on dist-test-slave-1nl6"
server_key: "5d209666db4cb765e5e26b5633305c43"
server_key_iv: "0b7d8a4a4bebc4d6fc3562762dab6c72"
server_key_version: "encryptionkey@0"
I20260808 17:00:39.697921 745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:39.725454 745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:39.727984 745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:39.728159 745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:39.728413 745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:00:39.728883 819 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 17:00:39.730966 745 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 17:00:39.731030 745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 17:00:39.731078 745 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 17:00:39.731736 745 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 17:00:39.731794 745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 17:00:39.731874 819 tablet_bootstrap.cc:492] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7: Bootstrap starting.
I20260808 17:00:39.739560 745 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:45261
I20260808 17:00:39.739984 745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 17:00:39.740481 964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:45261 every 8 connection(s)
I20260808 17:00:39.744133 819 log.cc:824] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7: Log is configured to *not* fsync() on all Append() calls
I20260808 17:00:39.746222 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 745
I20260808 17:00:39.746531 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:34407
--local_ip_for_outbound_sockets=127.8.14.66
--tserver_master_addrs=127.8.14.126:46633
--webserver_port=33525
--webserver_interface=127.8.14.66
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:00:39.747023 819 tablet_bootstrap.cc:492] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 17:00:39.747351 819 tablet_bootstrap.cc:492] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7: Bootstrap complete.
I20260808 17:00:39.748590 819 ts_tablet_manager.cc:1397] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.005s
I20260808 17:00:39.750204 819 raft_consensus.cc:348] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } } peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } }
I20260808 17:00:39.750520 819 raft_consensus.cc:729] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25d053f330d747329f158333e7295ac7, State: Initialized, Role: FOLLOWER
I20260808 17:00:39.750680 819 consensus_queue.cc:260] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } } peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } }
I20260808 17:00:39.751024 819 ts_tablet_manager.cc:1428] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
W20260808 17:00:39.797487 968 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:00:39.740723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:54727)
0808 17:00:39.740910 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:39.740954 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:39.741488 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:39.797251 (+ 55763us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
W20260808 17:00:39.797757 965 heartbeater.cc:644] Failed to heartbeat to 127.8.14.126:46633 (0 consecutive failures): Network error: Failed to ping master at 127.8.14.126:46633: Client connection negotiation failed: client connection to 127.8.14.126:46633: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 17:00:39.798110 968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:39.797993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:58337)
0808 17:00:39.798046 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:39.798090 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 17:00:39.798537 968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:39.798444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:39905)
0808 17:00:39.798496 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:39.798520 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 17:00:39.798664 965 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 17:00:39.841427 970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:39.841724 970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:39.841779 970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:39.843865 970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:39.843930 970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:39.843978 970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 17:00:39.846340 970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:34407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=33525
--tserver_master_addrs=127.8.14.126:46633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:39.846601 970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:39.846877 970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:39.854346 982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:39.854465 980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:39.854557 979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:39.854669 970 server_base.cc:1034] running on GCE node
I20260808 17:00:39.854990 970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:39.855283 970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:39.856433 970 hybrid_clock.cc:648] HybridClock initialized: now 1786208439856415 us; error 33 us; skew 500 ppm
Aug 08 17:00:39 dist-test-slave-1nl6 krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208439, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:00:39.859609 970 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 17:00:39.860098 970 webserver.cc:460] Webserver started at http://127.8.14.66:33525/ using document root <none> and password file <none>
I20260808 17:00:39.860340 970 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:39.860411 970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:39.862182 970 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 17:00:39.862941 991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:39.863116 970 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:00:39.863232 970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "a6d4e9cdda2240949054f7005002891c"
format_stamp: "Formatted at 2026-08-08 16:59:58 on dist-test-slave-1nl6"
server_key: "98d2ef0974494be6d4df7656980c2920"
server_key_iv: "dba150f6d4f7d2544d79cb9162b46d17"
server_key_version: "encryptionkey@0"
I20260808 17:00:39.863642 970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:39.879451 970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:39.881789 970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:39.881904 970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:39.882179 970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:00:39.882495 970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:00:39.882529 970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:39.882587 970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:00:39.882609 970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:39.888918 970 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:34407
I20260808 17:00:39.888963 1145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:34407 every 8 connection(s)
I20260808 17:00:39.889305 970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 17:00:39.890280 1152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:39.889925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:39627)
0808 17:00:39.890079 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:39.890146 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
W20260808 17:00:39.890690 1146 heartbeater.cc:644] Failed to heartbeat to 127.8.14.126:46633 (0 consecutive failures): Network error: Failed to ping master at 127.8.14.126:46633: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
I20260808 17:00:39.890977 1152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:39.890867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:56895)
0808 17:00:39.890928 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:39.890954 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 17:00:39.891255 1152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:39.891172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:45639)
0808 17:00:39.891214 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:39.891236 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 17:00:39.891366 1146 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 17:00:39.894212 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 970
I20260808 17:00:39.894508 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:45431
--local_ip_for_outbound_sockets=127.8.14.67
--tserver_master_addrs=127.8.14.126:46633
--webserver_port=43221
--webserver_interface=127.8.14.67
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:00:39.976912 1153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:39.977097 1153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:39.977118 1153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:39.978206 1153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:39.978247 1153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:39.978272 1153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 17:00:39.979790 1153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:45431
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=43221
--tserver_master_addrs=127.8.14.126:46633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:39.980026 1153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:39.980276 1153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:39.985236 1160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:39.985225 1158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:39.985225 1163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:39.985273 1153 server_base.cc:1034] running on GCE node
I20260808 17:00:39.985618 1153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:39.985867 1153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:39.986974 1153 hybrid_clock.cc:648] HybridClock initialized: now 1786208439986960 us; error 29 us; skew 500 ppm
Aug 08 17:00:39 dist-test-slave-1nl6 krb5kdc[31645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208439, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:00:39.989436 1153 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 17:00:39.989883 1153 webserver.cc:460] Webserver started at http://127.8.14.67:43221/ using document root <none> and password file <none>
I20260808 17:00:39.990130 1153 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:39.990203 1153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:39.991591 1153 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:00:39.992184 1177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:39.992386 1153 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:00:39.992492 1153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "5c20d9c140184f969e86ad6d3b0bad77"
format_stamp: "Formatted at 2026-08-08 16:59:58 on dist-test-slave-1nl6"
server_key: "6dc565364b867e034a4db7585bc26e74"
server_key_iv: "9ae6da08c25e0ffaefd6edcfa5fba1b8"
server_key_version: "encryptionkey@0"
I20260808 17:00:39.992831 1153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:40.020105 1153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:40.022581 1153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:40.022760 1153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:40.023044 1153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:00:40.023375 1153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:00:40.023432 1153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:40.023474 1153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:00:40.023519 1153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:40.029336 1153 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:45431
I20260808 17:00:40.029392 1329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:45431 every 8 connection(s)
I20260808 17:00:40.029726 1153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 17:00:40.030673 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 1153
I20260808 17:00:40.030727 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:40.030359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:52537)
0808 17:00:40.030516 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:40.030589 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
W20260808 17:00:40.031236 1330 heartbeater.cc:644] Failed to heartbeat to 127.8.14.126:46633 (0 consecutive failures): Network error: Failed to ping master at 127.8.14.126:46633: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
I20260808 17:00:40.031533 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:40.031437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:47565)
0808 17:00:40.031487 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:40.031511 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 17:00:40.031853 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:40.031757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:41495)
0808 17:00:40.031800 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:40.031826 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 17:00:40.031987 1330 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 17:00:40.799240 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:40.799016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:55697)
0808 17:00:40.799159 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:40.799190 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":60,"threads_started":1}
I20260808 17:00:40.891831 1335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:40.891626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:49671)
0808 17:00:40.891756 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:40.891784 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260808 17:00:41.032521 1336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:41.032313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:43919)
0808 17:00:41.032437 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:41.032471 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260808 17:00:41.408705 1337 raft_consensus.cc:482] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:00:41.408834 1337 raft_consensus.cc:504] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } } peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } }
I20260808 17:00:41.409199 1337 leader_election.cc:288] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407), 5c20d9c140184f969e86ad6d3b0bad77 (127.8.14.67:45431)
Aug 08 17:00:41 dist-test-slave-1nl6 krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208439, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.67@KRBTEST.COM
Aug 08 17:00:41 dist-test-slave-1nl6 krb5kdc[31645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208439, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.66@KRBTEST.COM
I20260808 17:00:41.420370 1340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:41.409386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:45431 (local address 127.8.14.65:59961)
0808 17:00:41.409525 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:41.409537 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:41.409599 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:41.410605 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:41.410606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:41.410880 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:00:41.411097 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:41.411103 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:41.412343 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:41.412344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:41.413182 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:41.413185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:41.413803 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:41.413804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:41.413925 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:41.414467 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:00:41.414475 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:00:41.416141 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:00:41.418662 (+ 2521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:41.418669 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:41.418671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:41.419041 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:41.419323 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:41.419324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:41.419324 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:41.419434 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:41.419823 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:00:41.419830 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:00:41.419969 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 17:00:41.420070 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 17:00:41.420262 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":60,"thread_start_us":77,"threads_started":1}
I20260808 17:00:41.420387 1343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:41.409508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:59961 (local address 127.8.14.67:45431)
0808 17:00:41.409641 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:41.409648 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:41.409683 (+ 35us) server_negotiation.cc:407] Connection header received
0808 17:00:41.410257 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:41.410259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:41.410312 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:41.410467 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:41.411260 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:41.412233 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:41.413288 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:41.413704 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:41.413841 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:41.416269 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:41.416273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:41.416273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:00:41.416284 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:41.418477 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:41.419136 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:41.419139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:41.419139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:41.419238 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:41.419511 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:41.419512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:41.419512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:41.419748 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:41.419842 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:41.420141 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 17:00:41.420267 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
I20260808 17:00:41.420699 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:41.409446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:36763 (local address 127.8.14.66:34407)
0808 17:00:41.409620 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:41.409642 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:41.409669 (+ 27us) server_negotiation.cc:407] Connection header received
0808 17:00:41.410248 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:41.410252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:41.410312 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:41.410467 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:41.411233 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:41.412106 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:41.413326 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:41.413751 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:41.413943 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:41.416673 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:41.416677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:41.416678 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:00:41.416691 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:41.418716 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:41.419186 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:41.419189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:41.419189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:41.419292 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:41.419534 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:41.419535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:41.419535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:41.419779 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:41.419888 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:41.420153 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 17:00:41.420319 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":55,"threads_started":1}
I20260808 17:00:41.420684 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:41.409302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:34407 (local address 127.8.14.65:36763)
0808 17:00:41.409433 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:41.409446 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:41.409512 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:41.410513 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:41.410516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:41.410880 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:00:41.411098 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:41.411104 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:41.412236 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:41.412237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:41.413174 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:41.413178 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:41.413880 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:41.413881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:41.413941 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:41.414634 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:00:41.414643 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:00:41.416568 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:00:41.418802 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:41.418803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:41.418804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:41.419087 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:41.419378 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:41.419379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:41.419380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:41.419456 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:41.419868 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:00:41.419869 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:00:41.419984 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 17:00:41.420072 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 17:00:41.420359 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":132,"thread_start_us":60,"threads_started":1}
W20260808 17:00:41.421339 812 leader_election.cc:341] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c20d9c140184f969e86ad6d3b0bad77 (127.8.14.67:45431): Not found: Tablet not found: 67446e3a4a0a4edfab0ae5a4b6eb3938
W20260808 17:00:41.421821 811 leader_election.cc:341] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407): Not found: Tablet not found: 67446e3a4a0a4edfab0ae5a4b6eb3938
I20260808 17:00:41.421881 811 leader_election.cc:302] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25d053f330d747329f158333e7295ac7; no voters: 5c20d9c140184f969e86ad6d3b0bad77, a6d4e9cdda2240949054f7005002891c
I20260808 17:00:41.422025 1337 raft_consensus.cc:2726] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:00:41.799820 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:41.799687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:42307)
0808 17:00:41.799744 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:41.799780 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 17:00:41.892518 1344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:41.892291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:56531)
0808 17:00:41.892428 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:41.892470 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":61,"threads_started":1}
I20260808 17:00:42.033140 1346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:42.032953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:44661)
0808 17:00:42.033069 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:42.033103 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260808 17:00:42.800521 1347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:42.800305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:50035)
0808 17:00:42.800445 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:42.800475 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260808 17:00:42.893143 1348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:42.892949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:39427)
0808 17:00:42.893067 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:42.893104 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260808 17:00:43.033759 1350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:43.033551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:46277)
0808 17:00:43.033684 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:43.033720 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260808 17:00:43.124797 1351 raft_consensus.cc:482] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:00:43.124904 1351 raft_consensus.cc:504] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } } peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } }
I20260808 17:00:43.125083 1351 leader_election.cc:288] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407), 5c20d9c140184f969e86ad6d3b0bad77 (127.8.14.67:45431)
W20260808 17:00:43.125491 812 leader_election.cc:341] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c20d9c140184f969e86ad6d3b0bad77 (127.8.14.67:45431): Not found: Tablet not found: 67446e3a4a0a4edfab0ae5a4b6eb3938
W20260808 17:00:43.125574 811 leader_election.cc:341] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407): Not found: Tablet not found: 67446e3a4a0a4edfab0ae5a4b6eb3938
I20260808 17:00:43.125674 811 leader_election.cc:302] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25d053f330d747329f158333e7295ac7; no voters: 5c20d9c140184f969e86ad6d3b0bad77, a6d4e9cdda2240949054f7005002891c
I20260808 17:00:43.125761 1351 raft_consensus.cc:2726] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:00:43.801159 1352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:43.800934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:35439)
0808 17:00:43.801064 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:43.801113 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260808 17:00:43.893817 1353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:43.893574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:42335)
0808 17:00:43.893711 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:43.893748 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":60,"threads_started":1}
I20260808 17:00:44.034471 1355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:44.034188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:44615)
0808 17:00:44.034397 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:44.034433 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":60,"threads_started":1}
I20260808 17:00:44.801805 1357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:44.801598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:49217)
0808 17:00:44.801721 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:44.801756 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260808 17:00:44.894466 1359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:44.894268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:54123)
0808 17:00:44.894389 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:44.894425 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260808 17:00:45.035212 1362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:45.034987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:34701)
0808 17:00:45.035138 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:45.035172 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
I20260808 17:00:45.802479 1363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:45.802257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:38831)
0808 17:00:45.802401 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:45.802433 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":57,"threads_started":1}
I20260808 17:00:45.895210 1367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:45.895019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:58423)
0808 17:00:45.895132 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:45.895167 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260808 17:00:46.036072 1371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:46.035817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:55125)
0808 17:00:46.035983 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:46.036030 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":70,"threads_started":1}
I20260808 17:00:46.270803 1372 raft_consensus.cc:482] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:00:46.271003 1372 raft_consensus.cc:504] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } } peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } }
I20260808 17:00:46.271291 1372 leader_election.cc:288] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407), 5c20d9c140184f969e86ad6d3b0bad77 (127.8.14.67:45431)
W20260808 17:00:46.271687 811 leader_election.cc:341] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407): Not found: Tablet not found: 67446e3a4a0a4edfab0ae5a4b6eb3938
W20260808 17:00:46.271761 812 leader_election.cc:341] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c20d9c140184f969e86ad6d3b0bad77 (127.8.14.67:45431): Not found: Tablet not found: 67446e3a4a0a4edfab0ae5a4b6eb3938
I20260808 17:00:46.271862 812 leader_election.cc:302] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25d053f330d747329f158333e7295ac7; no voters: 5c20d9c140184f969e86ad6d3b0bad77, a6d4e9cdda2240949054f7005002891c
I20260808 17:00:46.271970 1372 raft_consensus.cc:2726] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:00:46.803252 1373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:46.803022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:36211)
0808 17:00:46.803156 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:46.803201 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260808 17:00:46.895947 1374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:46.895707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:59725)
0808 17:00:46.895859 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:46.895904 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
I20260808 17:00:47.036828 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:47.036616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:34383)
0808 17:00:47.036750 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:47.036787 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260808 17:00:47.803891 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:47.803688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:34147)
0808 17:00:47.803809 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:47.803843 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260808 17:00:47.896673 1379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:47.896449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:37297)
0808 17:00:47.896590 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:47.896630 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
I20260808 17:00:48.037499 1381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:48.037296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:42699)
0808 17:00:48.037420 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:48.037458 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260808 17:00:48.804565 1382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:48.804362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:53211)
0808 17:00:48.804482 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:48.804518 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260808 17:00:48.897318 1383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:48.897112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:38875)
0808 17:00:48.897237 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:48.897274 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
I20260808 17:00:49.038096 1384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:49.037901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:43595)
0808 17:00:49.038030 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:49.038060 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260808 17:00:49.363229 1385 raft_consensus.cc:482] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:00:49.363333 1385 raft_consensus.cc:504] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } } peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } }
I20260808 17:00:49.363554 1385 leader_election.cc:288] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407), 5c20d9c140184f969e86ad6d3b0bad77 (127.8.14.67:45431)
W20260808 17:00:49.363996 811 leader_election.cc:341] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407): Not found: Tablet not found: 67446e3a4a0a4edfab0ae5a4b6eb3938
W20260808 17:00:49.364074 812 leader_election.cc:341] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c20d9c140184f969e86ad6d3b0bad77 (127.8.14.67:45431): Not found: Tablet not found: 67446e3a4a0a4edfab0ae5a4b6eb3938
I20260808 17:00:49.364122 812 leader_election.cc:302] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25d053f330d747329f158333e7295ac7; no voters: 5c20d9c140184f969e86ad6d3b0bad77, a6d4e9cdda2240949054f7005002891c
I20260808 17:00:49.364223 1385 raft_consensus.cc:2726] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:00:49.805234 1386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:49.805007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:38199)
0808 17:00:49.805139 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:49.805182 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
I20260808 17:00:49.898030 1387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:49.897819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:43211)
0808 17:00:49.897941 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:49.897976 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
I20260808 17:00:50.038933 1388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:50.038663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:41485)
0808 17:00:50.038792 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:50.038849 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260808 17:00:50.805869 1389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:50.805675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:45783)
0808 17:00:50.805792 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:50.805821 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260808 17:00:50.898576 1392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:50.898393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:50951)
0808 17:00:50.898506 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:50.898538 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260808 17:00:51.039577 1393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:51.039374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:36001)
0808 17:00:51.039505 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:51.039535 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260808 17:00:51.806550 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:51.806335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:46279)
0808 17:00:51.806464 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:51.806501 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260808 17:00:51.899237 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:51.899030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:34819)
0808 17:00:51.899168 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:51.899197 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260808 17:00:52.040174 1398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:52.039967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:37127)
0808 17:00:52.040103 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:52.040134 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
I20260808 17:00:52.640674 1400 raft_consensus.cc:482] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:00:52.640779 1400 raft_consensus.cc:504] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6d4e9cdda2240949054f7005002891c" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 34407 } } peers { permanent_uuid: "5c20d9c140184f969e86ad6d3b0bad77" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45431 } } peers { permanent_uuid: "25d053f330d747329f158333e7295ac7" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 45261 } }
I20260808 17:00:52.640992 1400 leader_election.cc:288] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407), 5c20d9c140184f969e86ad6d3b0bad77 (127.8.14.67:45431)
W20260808 17:00:52.641373 812 leader_election.cc:341] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c20d9c140184f969e86ad6d3b0bad77 (127.8.14.67:45431): Not found: Tablet not found: 67446e3a4a0a4edfab0ae5a4b6eb3938
W20260808 17:00:52.641484 811 leader_election.cc:341] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6d4e9cdda2240949054f7005002891c (127.8.14.66:34407): Not found: Tablet not found: 67446e3a4a0a4edfab0ae5a4b6eb3938
I20260808 17:00:52.641521 811 leader_election.cc:302] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 25d053f330d747329f158333e7295ac7; no voters: 5c20d9c140184f969e86ad6d3b0bad77, a6d4e9cdda2240949054f7005002891c
I20260808 17:00:52.641616 1400 raft_consensus.cc:2726] T 67446e3a4a0a4edfab0ae5a4b6eb3938 P 25d053f330d747329f158333e7295ac7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 17:00:52.807387 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:52.807127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:43853)
0808 17:00:52.807292 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:52.807339 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":75,"threads_started":1}
I20260808 17:00:52.899852 1403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:52.899633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:35915)
0808 17:00:52.899767 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:52.899810 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260808 17:00:53.040855 1404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:53.040624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:43315)
0808 17:00:53.040772 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:53.040814 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260808 17:00:53.808054 1405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:53.807828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:44259)
0808 17:00:53.807970 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:53.808004 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":69,"threads_started":1}
I20260808 17:00:53.900626 1406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:53.900403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:45411)
0808 17:00:53.900535 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:53.900580 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260808 17:00:54.041550 1407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:54.041309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:58467)
0808 17:00:54.041463 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:54.041502 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260808 17:00:54.808892 1408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:54.808663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.65:48029)
0808 17:00:54.808803 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:54.808833 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260808 17:00:54.901379 1409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:54.901131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.66:44685)
0808 17:00:54.901304 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:54.901338 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":78,"threads_started":1}
I20260808 17:00:55.035735 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 745
I20260808 17:00:55.042326 1410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:55.042124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46633 (local address 127.8.14.67:49037)
0808 17:00:55.042252 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:55.042283 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.126:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
I20260808 17:00:55.044808 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 970
I20260808 17:00:55.055228 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 1153
2026-08-08T17:00:55Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58762 ms)
[----------] 30 tests from SecurityITest (264626 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:00:55 dist-test-slave-1nl6 krb5kdc[1417](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:00:55 dist-test-slave-1nl6 krb5kdc[1417](info): set up 2 sockets
Aug 08 17:00:55 dist-test-slave-1nl6 krb5kdc[1417](info): commencing operation
krb5kdc: starting...
W20260808 17:00:56.294687 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:00:56 dist-test-slave-1nl6 krb5kdc[1417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:00:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:00:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 17:00:56.411841 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:44223
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:44155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:44223
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:00:56.494362 1437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:56.494596 1437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:56.494633 1437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:56.495812 1437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:00:56.495882 1437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:56.495905 1437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:56.495926 1437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:00:56.495955 1437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:00:56.497411 1437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:44155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:44223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:44223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:56.497681 1437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:56.497905 1437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:56.502770 1446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:56.502714 1447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:56.502707 1450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:56.503072 1437 server_base.cc:1034] running on GCE node
I20260808 17:00:56.503329 1437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:56.503652 1437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:56.504825 1437 hybrid_clock.cc:648] HybridClock initialized: now 1786208456504819 us; error 37 us; skew 500 ppm
Aug 08 17:00:56 dist-test-slave-1nl6 krb5kdc[1417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208456, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:00:56.507462 1437 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 17:00:56.507934 1437 webserver.cc:460] Webserver started at http://127.8.14.126:37353/ using document root <none> and password file <none>
I20260808 17:00:56.508180 1437 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:56.508252 1437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:56.508404 1437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:00:56.509423 1437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "ccda69589d23446786b5f3f4123f22af"
format_stamp: "Formatted at 2026-08-08 17:00:56 on dist-test-slave-1nl6"
server_key: "fa3e25003416c5d6ba2fad68fac9f495"
server_key_iv: "02478722b7a0a4c76042a0e31e8d0214"
server_key_version: "encryptionkey@0"
I20260808 17:00:56.509795 1437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "ccda69589d23446786b5f3f4123f22af"
format_stamp: "Formatted at 2026-08-08 17:00:56 on dist-test-slave-1nl6"
server_key: "fa3e25003416c5d6ba2fad68fac9f495"
server_key_iv: "02478722b7a0a4c76042a0e31e8d0214"
server_key_version: "encryptionkey@0"
I20260808 17:00:56.511248 1437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:00:56.512027 1457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:56.512236 1437 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:00:56.512331 1437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "ccda69589d23446786b5f3f4123f22af"
format_stamp: "Formatted at 2026-08-08 17:00:56 on dist-test-slave-1nl6"
server_key: "fa3e25003416c5d6ba2fad68fac9f495"
server_key_iv: "02478722b7a0a4c76042a0e31e8d0214"
server_key_version: "encryptionkey@0"
I20260808 17:00:56.512413 1437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:56.521752 1437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:56.524115 1437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:56.524287 1437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:56.528666 1437 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:44223
I20260808 17:00:56.528698 1532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:44223 every 8 connection(s)
I20260808 17:00:56.529047 1437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:00:56.529920 1534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:00:56.532231 1534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af: Bootstrap starting.
I20260808 17:00:56.532766 1534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af: Neither blocks nor log segments found. Creating new log.
I20260808 17:00:56.533010 1534 log.cc:824] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af: Log is configured to *not* fsync() on all Append() calls
I20260808 17:00:56.533658 1534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af: No bootstrap required, opened a new log
I20260808 17:00:56.535220 1534 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccda69589d23446786b5f3f4123f22af" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44223 } }
I20260808 17:00:56.535356 1534 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:00:56.535379 1534 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccda69589d23446786b5f3f4123f22af, State: Initialized, Role: FOLLOWER
I20260808 17:00:56.535477 1534 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccda69589d23446786b5f3f4123f22af" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44223 } }
I20260808 17:00:56.535559 1534 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:00:56.535597 1534 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:00:56.535650 1534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:00:56.536182 1534 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccda69589d23446786b5f3f4123f22af" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44223 } }
I20260808 17:00:56.536324 1534 leader_election.cc:302] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccda69589d23446786b5f3f4123f22af; no voters:
I20260808 17:00:56.536448 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 1437
I20260808 17:00:56.536525 1534 leader_election.cc:288] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:00:56.536580 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:00:56.536556 1537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:00:56.536759 1537 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [term 1 LEADER]: Becoming Leader. State: Replica: ccda69589d23446786b5f3f4123f22af, State: Running, Role: LEADER
I20260808 17:00:56.536839 8249 external_mini_cluster.cc:1428] Setting key d0140f2a1e3ceffc90058742d0e3debf
I20260808 17:00:56.536859 1534 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:00:56.536899 1537 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccda69589d23446786b5f3f4123f22af" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44223 } }
I20260808 17:00:56.537325 1537 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccda69589d23446786b5f3f4123f22af. Latest consensus state: current_term: 1 leader_uuid: "ccda69589d23446786b5f3f4123f22af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccda69589d23446786b5f3f4123f22af" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44223 } } }
I20260808 17:00:56.537513 1537 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [sys.catalog]: This master's current role is: LEADER
I20260808 17:00:56.537623 1538 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccda69589d23446786b5f3f4123f22af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccda69589d23446786b5f3f4123f22af" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 44223 } } }
I20260808 17:00:56.537721 1538 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af [sys.catalog]: This master's current role is: LEADER
I20260808 17:00:56.537981 1550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:00:56.538581 1550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:00:56.540228 1550 catalog_manager.cc:1347] Generated new cluster ID: 7796d7cf6f8641d497a4534b075a495e
I20260808 17:00:56.540292 1550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:00:56 dist-test-slave-1nl6 krb5kdc[1417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:00:56.549539 1551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:56.537920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34868 (local address 127.8.14.126:44223)
0808 17:00:56.538124 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:56.538134 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:56.538166 (+ 32us) server_negotiation.cc:407] Connection header received
0808 17:00:56.538282 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:56.538290 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:56.538364 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:56.539296 (+ 932us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:56.539405 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:56.540721 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:56.541803 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:56.542443 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:56.542914 (+ 471us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:56.545475 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:56.545480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:56.545480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:00:56.545492 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:56.547546 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:56.547944 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:56.547946 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:56.547946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:56.548063 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:56.548280 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:56.548280 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:56.548281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:56.548573 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:56.549130 (+ 557us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:56.549199 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 17:00:56.549331 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":95,"threads_started":1}
I20260808 17:00:56.554032 1550 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:00:56.554708 1550 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:00:56.560148 1550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af: Generated new TSK 0
I20260808 17:00:56.560335 1550 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 17:00:56.591069 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:44223
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:44155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:00:56.672943 1567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:56.673180 1567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:56.673219 1567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:56.674367 1567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:56.674440 1567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:56.674490 1567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 17:00:56.676164 1567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:44155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:44223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:56.676437 1567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:56.676669 1567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:56.681828 1575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:56.681843 1576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:56.681875 1580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:56.681962 1567 server_base.cc:1034] running on GCE node
I20260808 17:00:56.682309 1567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:56.682560 1567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:56.683722 1567 hybrid_clock.cc:648] HybridClock initialized: now 1786208456683716 us; error 36 us; skew 500 ppm
Aug 08 17:00:56 dist-test-slave-1nl6 krb5kdc[1417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208456, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:00:56.686220 1567 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 17:00:56.686590 1567 webserver.cc:460] Webserver started at http://127.8.14.65:35567/ using document root <none> and password file <none>
I20260808 17:00:56.686764 1567 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:56.686803 1567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:56.686954 1567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:00:56.687960 1567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "98bd056da6de4d28ad38e94459a232de"
format_stamp: "Formatted at 2026-08-08 17:00:56 on dist-test-slave-1nl6"
server_key: "61ae6dd3375989d4713b3facf8172cec"
server_key_iv: "0454b27139c935c74f41be3544fee2cc"
server_key_version: "encryptionkey@0"
I20260808 17:00:56.688264 1567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "98bd056da6de4d28ad38e94459a232de"
format_stamp: "Formatted at 2026-08-08 17:00:56 on dist-test-slave-1nl6"
server_key: "61ae6dd3375989d4713b3facf8172cec"
server_key_iv: "0454b27139c935c74f41be3544fee2cc"
server_key_version: "encryptionkey@0"
I20260808 17:00:56.689514 1567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:00:56.690196 1590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:56.690357 1567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:00:56.690447 1567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "98bd056da6de4d28ad38e94459a232de"
format_stamp: "Formatted at 2026-08-08 17:00:56 on dist-test-slave-1nl6"
server_key: "61ae6dd3375989d4713b3facf8172cec"
server_key_iv: "0454b27139c935c74f41be3544fee2cc"
server_key_version: "encryptionkey@0"
I20260808 17:00:56.690501 1567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:56.702118 1567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:56.703917 1567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:56.704027 1567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:56.704203 1567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:00:56.704515 1567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:00:56.704551 1567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:56.704571 1567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:00:56.704619 1567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:56.710407 1567 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:34849
I20260808 17:00:56.710481 1737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:34849 every 8 connection(s)
I20260808 17:00:56.710785 1567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 17:00:56.716207 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 1567
I20260808 17:00:56.716326 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 17:00:56.716529 8249 external_mini_cluster.cc:1428] Setting key 4b8447f91d73a3fe5b111586d23d06c6
Aug 08 17:00:56 dist-test-slave-1nl6 krb5kdc[1417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208456, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:00:56.720120 1740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:56.711535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44223 (local address 127.8.14.65:39837)
0808 17:00:56.711697 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:56.711739 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:56.712213 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:56.712415 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:56.712418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:56.712975 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:00:56.713158 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:56.713162 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:56.713814 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:56.713814 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:56.714596 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:56.714598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:56.715015 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:56.715016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:56.715115 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:56.715526 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:00:56.715534 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:00:56.717154 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:00:56.718788 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:56.718803 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:56.718806 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:56.719097 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:56.719304 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:56.719304 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:56.719304 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:56.719388 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:56.719657 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:00:56.719662 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:00:56.719767 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 17:00:56.719855 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 17:00:56.719967 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260808 17:00:56.720130 1551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:56.711587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:39837 (local address 127.8.14.126:44223)
0808 17:00:56.711648 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:56.711650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:56.711831 (+ 181us) server_negotiation.cc:407] Connection header received
0808 17:00:56.712293 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:56.712294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:56.712306 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:56.712365 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:56.713233 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:56.713725 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:56.714686 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:56.714929 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:56.714978 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:56.717227 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:56.717228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:56.717229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:00:56.717236 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:56.718657 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:56.719167 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:56.719168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:56.719169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:56.719236 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:56.719445 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:56.719446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:56.719447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:56.719586 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:56.719660 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:56.719902 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 17:00:56.720003 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 17:00:56.720942 1738 heartbeater.cc:347] Connected to a master server at 127.8.14.126:44223
I20260808 17:00:56.721045 1738 heartbeater.cc:464] Registering TS with master...
I20260808 17:00:56.721232 1738 heartbeater.cc:511] Master 127.8.14.126:44223 requested a full tablet report, sending...
I20260808 17:00:56.721839 1484 ts_manager.cc:194] Registered new tserver with Master: 98bd056da6de4d28ad38e94459a232de (127.8.14.65:34849)
I20260808 17:00:56.722433 1484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:39837
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 17:00:56.741837 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:44223
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:44155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:00:56.824497 1745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:56.824743 1745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:56.824786 1745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:56.826035 1745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:56.826124 1745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:56.826175 1745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 17:00:56.827890 1745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:44155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:44223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:56.828184 1745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:56.828430 1745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:56.833863 1750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:56.833873 1754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:56.833878 1751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:56.834134 1745 server_base.cc:1034] running on GCE node
I20260808 17:00:56.834326 1745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:56.834517 1745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:56.835650 1745 hybrid_clock.cc:648] HybridClock initialized: now 1786208456835640 us; error 26 us; skew 500 ppm
Aug 08 17:00:56 dist-test-slave-1nl6 krb5kdc[1417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208456, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:00:56.838339 1745 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 17:00:56.838778 1745 webserver.cc:460] Webserver started at http://127.8.14.66:39415/ using document root <none> and password file <none>
I20260808 17:00:56.839026 1745 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:56.839075 1745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:56.839169 1745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:00:56.840144 1745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "dfdcb91b9fb942f591d2cccaa6bc8b25"
format_stamp: "Formatted at 2026-08-08 17:00:56 on dist-test-slave-1nl6"
server_key: "2c1b52ce1ec8b18ea54f6a16bf17e78f"
server_key_iv: "7ce27cf6da06ea69dd4354f293eaceb2"
server_key_version: "encryptionkey@0"
I20260808 17:00:56.840456 1745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "dfdcb91b9fb942f591d2cccaa6bc8b25"
format_stamp: "Formatted at 2026-08-08 17:00:56 on dist-test-slave-1nl6"
server_key: "2c1b52ce1ec8b18ea54f6a16bf17e78f"
server_key_iv: "7ce27cf6da06ea69dd4354f293eaceb2"
server_key_version: "encryptionkey@0"
I20260808 17:00:56.841737 1745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:00:56.842391 1766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:56.842579 1745 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:00:56.842662 1745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "dfdcb91b9fb942f591d2cccaa6bc8b25"
format_stamp: "Formatted at 2026-08-08 17:00:56 on dist-test-slave-1nl6"
server_key: "2c1b52ce1ec8b18ea54f6a16bf17e78f"
server_key_iv: "7ce27cf6da06ea69dd4354f293eaceb2"
server_key_version: "encryptionkey@0"
I20260808 17:00:56.842736 1745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:56.863631 1745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:56.865456 1745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:56.865569 1745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:56.865767 1745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:00:56.866019 1745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:00:56.866050 1745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:56.866072 1745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:00:56.866086 1745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:56.871762 1745 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:36701
I20260808 17:00:56.871876 1916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:36701 every 8 connection(s)
I20260808 17:00:56.872184 1745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 17:00:56.877287 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 1745
I20260808 17:00:56.877595 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 17:00:56.877923 8249 external_mini_cluster.cc:1428] Setting key 063178e434e29ba48f65403c953dcda5
Aug 08 17:00:56 dist-test-slave-1nl6 krb5kdc[1417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208456, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:00:56.882174 1921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:56.872918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44223 (local address 127.8.14.66:53309)
0808 17:00:56.873113 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:56.873148 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:56.873615 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:56.873843 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:56.873846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:56.874409 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:00:56.874593 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:56.874597 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:56.875298 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:56.875299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:56.876214 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:56.876217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:56.876564 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:56.876565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:56.876662 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:56.877107 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:00:56.877115 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:00:56.878919 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:00:56.880726 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:56.880733 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:56.880734 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:56.880949 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:56.881169 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:56.881170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:56.881170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:56.881269 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:56.881664 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:00:56.881670 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:00:56.881786 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 17:00:56.881898 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 17:00:56.882020 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":110,"threads_started":1}
I20260808 17:00:56.882180 1551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:56.872936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:53309 (local address 127.8.14.126:44223)
0808 17:00:56.872994 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:56.872997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:56.873236 (+ 239us) server_negotiation.cc:407] Connection header received
0808 17:00:56.873699 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:56.873700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:56.873711 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:56.873763 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:56.874672 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:56.875218 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:56.876292 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:56.876486 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:56.876543 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:56.879019 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:56.879020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:56.879021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:00:56.879028 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:56.880636 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:56.881031 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:56.881032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:56.881032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:56.881099 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:56.881347 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:56.881348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:56.881348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:56.881502 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:56.881578 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:56.881949 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 17:00:56.882046 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:00:56.882697 1917 heartbeater.cc:347] Connected to a master server at 127.8.14.126:44223
I20260808 17:00:56.882815 1917 heartbeater.cc:464] Registering TS with master...
I20260808 17:00:56.883039 1917 heartbeater.cc:511] Master 127.8.14.126:44223 requested a full tablet report, sending...
I20260808 17:00:56.883471 1484 ts_manager.cc:194] Registered new tserver with Master: dfdcb91b9fb942f591d2cccaa6bc8b25 (127.8.14.66:36701)
I20260808 17:00:56.883997 1484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:53309
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 17:00:56.902889 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:44223
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:44155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:00:56.984313 1924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:56.984510 1924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:56.984534 1924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:56.985605 1924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:56.985642 1924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:56.985718 1924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 17:00:56.987107 1924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:44155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:44223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:56.987339 1924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:56.987591 1924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:56.992636 1936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:56.992637 1931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:56.992787 1930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:56.992882 1924 server_base.cc:1034] running on GCE node
I20260808 17:00:56.993088 1924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:56.993395 1924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:56.994560 1924 hybrid_clock.cc:648] HybridClock initialized: now 1786208456994544 us; error 32 us; skew 500 ppm
Aug 08 17:00:56 dist-test-slave-1nl6 krb5kdc[1417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208456, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:00:56.997006 1924 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 17:00:56.997447 1924 webserver.cc:460] Webserver started at http://127.8.14.67:37381/ using document root <none> and password file <none>
I20260808 17:00:56.997628 1924 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:56.997674 1924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:56.997764 1924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:00:56.998731 1924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "9a9e390c492844e6912aa8fd56a018b5"
format_stamp: "Formatted at 2026-08-08 17:00:56 on dist-test-slave-1nl6"
server_key: "8b198e4a440ed87e61369bc4c6b641e8"
server_key_iv: "5863ff18bda8a8441fe01cbeae63db13"
server_key_version: "encryptionkey@0"
I20260808 17:00:56.999119 1924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "9a9e390c492844e6912aa8fd56a018b5"
format_stamp: "Formatted at 2026-08-08 17:00:56 on dist-test-slave-1nl6"
server_key: "8b198e4a440ed87e61369bc4c6b641e8"
server_key_iv: "5863ff18bda8a8441fe01cbeae63db13"
server_key_version: "encryptionkey@0"
I20260808 17:00:57.000360 1924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:00:57.001035 1944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:57.001209 1924 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:00:57.001319 1924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "9a9e390c492844e6912aa8fd56a018b5"
format_stamp: "Formatted at 2026-08-08 17:00:56 on dist-test-slave-1nl6"
server_key: "8b198e4a440ed87e61369bc4c6b641e8"
server_key_iv: "5863ff18bda8a8441fe01cbeae63db13"
server_key_version: "encryptionkey@0"
I20260808 17:00:57.001397 1924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:57.010170 1924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:57.012089 1924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:57.012274 1924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:57.012507 1924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:00:57.012861 1924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:00:57.012920 1924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:57.012962 1924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:00:57.012990 1924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:57.019073 1924 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:44493
I20260808 17:00:57.019111 2108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:44493 every 8 connection(s)
I20260808 17:00:57.019456 1924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:00:57 dist-test-slave-1nl6 krb5kdc[1417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208456, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:00:57.028010 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 1924
I20260808 17:00:57.028173 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 17:00:57.028436 8249 external_mini_cluster.cc:1428] Setting key a133a4606e24f2544b1cb1eeec9c6bc2
I20260808 17:00:57.028905 1551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.020201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:37727 (local address 127.8.14.126:44223)
0808 17:00:57.020287 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:57.020290 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:57.020441 (+ 151us) server_negotiation.cc:407] Connection header received
0808 17:00:57.020919 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:57.020920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:57.020930 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:57.020984 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:57.021902 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.022390 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.023454 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.023639 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.023687 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.025899 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:57.025901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:57.025901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:00:57.025909 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:57.027265 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:57.027699 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:57.027700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:57.027700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:57.027773 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:57.028080 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:57.028081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:57.028100 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:57.028273 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:57.028344 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:57.028698 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 17:00:57.028793 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 17:00:57.029134 2111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.020147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:44223 (local address 127.8.14.67:37727)
0808 17:00:57.020314 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:57.020353 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:57.020826 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:57.021040 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:57.021043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:57.021621 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:00:57.021815 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.021820 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.022473 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.022474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.023369 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.023373 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.023710 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.023711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.023809 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.024295 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:00:57.024304 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:00:57.025790 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:00:57.027366 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:57.027373 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:57.027374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:57.027629 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:57.027843 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:57.027844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:57.027845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:57.028000 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:57.028337 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:00:57.028345 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:00:57.028524 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 17:00:57.028649 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 17:00:57.028845 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":104,"threads_started":1}
I20260808 17:00:57.029584 2109 heartbeater.cc:347] Connected to a master server at 127.8.14.126:44223
I20260808 17:00:57.029701 2109 heartbeater.cc:464] Registering TS with master...
I20260808 17:00:57.029892 2109 heartbeater.cc:511] Master 127.8.14.126:44223 requested a full tablet report, sending...
I20260808 17:00:57.030296 1484 ts_manager.cc:194] Registered new tserver with Master: 9a9e390c492844e6912aa8fd56a018b5 (127.8.14.67:44493)
I20260808 17:00:57.030690 1484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:37727
I20260808 17:00:57.040453 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:00:57.047686 1551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.041871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34876 (local address 127.8.14.126:44223)
0808 17:00:57.041948 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:57.041950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:57.041979 (+ 29us) server_negotiation.cc:407] Connection header received
0808 17:00:57.042043 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:57.042044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:57.042053 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:57.042099 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:57.042631 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.043160 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.043867 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.044044 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.044093 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.045173 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:57.045174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:57.045174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:00:57.045180 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:57.046690 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:57.047031 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:57.047032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:57.047033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:57.047082 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:57.047228 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:57.047228 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:57.047228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:57.047368 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:57.047437 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:57.047534 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 17:00:57.047599 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 17:00:57.049119 1484 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:00:57.050709 1484 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:00:57.057192 2124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.053174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:44493 (local address 127.0.0.1:38988)
0808 17:00:57.053449 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:57.053458 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:57.053505 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:57.053731 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:57.053734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:57.053737 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:00:57.053802 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.053804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.054727 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.054728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.055967 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.055970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.056868 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.056870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.056971 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.056989 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 17:00:57.057055 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 17:00:57.057110 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":44,"threads_started":1}
I20260808 17:00:57.057322 2126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.053320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36434 (local address 127.8.14.65:34849)
0808 17:00:57.053573 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:57.053578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:57.053589 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:00:57.053657 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:57.053660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:57.053677 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:57.053926 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:00:57.054007 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.055140 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.056362 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.056889 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.056996 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.057058 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:57.057178 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 17:00:57.057225 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":115,"threads_started":1}
I20260808 17:00:57.057308 2120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.052986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:34849 (local address 127.0.0.1:36434)
0808 17:00:57.053285 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:57.053297 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:57.053350 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:57.053789 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:57.053790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:57.053793 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:00:57.053845 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.053847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.055251 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.055252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.056270 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.056272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.057021 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.057022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.057087 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.057092 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 17:00:57.057153 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 17:00:57.057194 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":28,"thread_start_us":56,"threads_started":1}
I20260808 17:00:57.057400 2119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.052962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:36701 (local address 127.0.0.1:43900)
0808 17:00:57.053130 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:57.053178 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:57.053245 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:57.053808 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:57.053808 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:57.053810 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:00:57.053849 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.053850 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.055270 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.055271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.056313 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.056316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.057178 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.057178 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.057233 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.057236 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:00:57.057266 (+ 30us) client_negotiation.cc:240] Negotiation successful
0808 17:00:57.057297 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":37,"thread_start_us":57,"threads_started":1}
I20260808 17:00:57.057686 2125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.053267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38988 (local address 127.8.14.67:44493)
0808 17:00:57.053429 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:57.053433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:57.053499 (+ 66us) server_negotiation.cc:407] Connection header received
0808 17:00:57.053574 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:57.053576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:57.053588 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:57.053890 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:00:57.053959 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.054624 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.056055 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.056775 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.056851 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.056907 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:57.057575 (+ 668us) server_negotiation.cc:299] Negotiation successful
0808 17:00:57.057620 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260808 17:00:57.057760 2122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.053074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43900 (local address 127.8.14.66:36701)
0808 17:00:57.053642 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:57.053647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:57.053656 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:00:57.053694 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:57.053695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:57.053712 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:57.053926 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:00:57.054008 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.055130 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.056407 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.057089 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.057456 (+ 367us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.057526 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:57.057606 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 17:00:57.057652 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":97,"threads_started":1}
I20260808 17:00:57.058424 1829 tablet_service.cc:1467] Processing CreateTablet for tablet b25664e58680497090817eced978d97f (DEFAULT_TABLE table=test-table [id=384303ddb77d4bbd8ff2b0ee812a4955]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:00:57.058436 2015 tablet_service.cc:1467] Processing CreateTablet for tablet b25664e58680497090817eced978d97f (DEFAULT_TABLE table=test-table [id=384303ddb77d4bbd8ff2b0ee812a4955]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:00:57.058563 1649 tablet_service.cc:1467] Processing CreateTablet for tablet b25664e58680497090817eced978d97f (DEFAULT_TABLE table=test-table [id=384303ddb77d4bbd8ff2b0ee812a4955]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:00:57.058796 2015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b25664e58680497090817eced978d97f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:00:57.058805 1829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b25664e58680497090817eced978d97f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:00:57.058869 1649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b25664e58680497090817eced978d97f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:00:57.061647 2127 tablet_bootstrap.cc:492] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25: Bootstrap starting.
I20260808 17:00:57.062074 2129 tablet_bootstrap.cc:492] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de: Bootstrap starting.
I20260808 17:00:57.062345 2127 tablet_bootstrap.cc:654] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25: Neither blocks nor log segments found. Creating new log.
I20260808 17:00:57.062783 2129 tablet_bootstrap.cc:654] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de: Neither blocks nor log segments found. Creating new log.
I20260808 17:00:57.062976 2127 log.cc:824] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25: Log is configured to *not* fsync() on all Append() calls
I20260808 17:00:57.063037 2128 tablet_bootstrap.cc:492] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5: Bootstrap starting.
I20260808 17:00:57.063396 2129 log.cc:824] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de: Log is configured to *not* fsync() on all Append() calls
I20260808 17:00:57.063735 2128 tablet_bootstrap.cc:654] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5: Neither blocks nor log segments found. Creating new log.
I20260808 17:00:57.063846 2127 tablet_bootstrap.cc:492] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25: No bootstrap required, opened a new log
I20260808 17:00:57.063927 2127 ts_tablet_manager.cc:1397] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:00:57.064149 2129 tablet_bootstrap.cc:492] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de: No bootstrap required, opened a new log
I20260808 17:00:57.064342 2128 log.cc:824] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5: Log is configured to *not* fsync() on all Append() calls
I20260808 17:00:57.064424 2129 ts_tablet_manager.cc:1397] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:00:57.065163 2128 tablet_bootstrap.cc:492] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5: No bootstrap required, opened a new log
I20260808 17:00:57.065241 2128 ts_tablet_manager.cc:1397] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:00:57.066038 2129 raft_consensus.cc:348] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfdcb91b9fb942f591d2cccaa6bc8b25" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 36701 } } peers { permanent_uuid: "98bd056da6de4d28ad38e94459a232de" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 34849 } } peers { permanent_uuid: "9a9e390c492844e6912aa8fd56a018b5" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44493 } }
I20260808 17:00:57.065991 2127 raft_consensus.cc:348] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfdcb91b9fb942f591d2cccaa6bc8b25" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 36701 } } peers { permanent_uuid: "98bd056da6de4d28ad38e94459a232de" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 34849 } } peers { permanent_uuid: "9a9e390c492844e6912aa8fd56a018b5" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44493 } }
I20260808 17:00:57.066197 2127 raft_consensus.cc:374] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:00:57.066197 2129 raft_consensus.cc:374] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:00:57.066246 2129 raft_consensus.cc:729] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98bd056da6de4d28ad38e94459a232de, State: Initialized, Role: FOLLOWER
I20260808 17:00:57.066246 2127 raft_consensus.cc:729] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfdcb91b9fb942f591d2cccaa6bc8b25, State: Initialized, Role: FOLLOWER
I20260808 17:00:57.066387 2129 consensus_queue.cc:260] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfdcb91b9fb942f591d2cccaa6bc8b25" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 36701 } } peers { permanent_uuid: "98bd056da6de4d28ad38e94459a232de" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 34849 } } peers { permanent_uuid: "9a9e390c492844e6912aa8fd56a018b5" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44493 } }
I20260808 17:00:57.066388 2127 consensus_queue.cc:260] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfdcb91b9fb942f591d2cccaa6bc8b25" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 36701 } } peers { permanent_uuid: "98bd056da6de4d28ad38e94459a232de" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 34849 } } peers { permanent_uuid: "9a9e390c492844e6912aa8fd56a018b5" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44493 } }
I20260808 17:00:57.066741 1917 heartbeater.cc:503] Master 127.8.14.126:44223 was elected leader, sending a full tablet report...
I20260808 17:00:57.066740 2129 ts_tablet_manager.cc:1428] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:00:57.066890 1738 heartbeater.cc:503] Master 127.8.14.126:44223 was elected leader, sending a full tablet report...
I20260808 17:00:57.067025 2127 ts_tablet_manager.cc:1428] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 17:00:57.067642 2128 raft_consensus.cc:348] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfdcb91b9fb942f591d2cccaa6bc8b25" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 36701 } } peers { permanent_uuid: "98bd056da6de4d28ad38e94459a232de" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 34849 } } peers { permanent_uuid: "9a9e390c492844e6912aa8fd56a018b5" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44493 } }
I20260808 17:00:57.067821 2128 raft_consensus.cc:374] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:00:57.067873 2128 raft_consensus.cc:729] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a9e390c492844e6912aa8fd56a018b5, State: Initialized, Role: FOLLOWER
I20260808 17:00:57.067983 2128 consensus_queue.cc:260] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfdcb91b9fb942f591d2cccaa6bc8b25" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 36701 } } peers { permanent_uuid: "98bd056da6de4d28ad38e94459a232de" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 34849 } } peers { permanent_uuid: "9a9e390c492844e6912aa8fd56a018b5" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44493 } }
I20260808 17:00:57.068239 2128 ts_tablet_manager.cc:1428] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:00:57.068285 2109 heartbeater.cc:503] Master 127.8.14.126:44223 was elected leader, sending a full tablet report...
W20260808 17:00:57.122792 1919 tablet.cc:2369] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:00:57.211660 1739 tablet.cc:2369] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:00:57.235144 2138 raft_consensus.cc:482] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:00:57.235306 2138 raft_consensus.cc:504] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfdcb91b9fb942f591d2cccaa6bc8b25" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 36701 } } peers { permanent_uuid: "98bd056da6de4d28ad38e94459a232de" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 34849 } } peers { permanent_uuid: "9a9e390c492844e6912aa8fd56a018b5" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44493 } }
I20260808 17:00:57.235682 2138 leader_election.cc:288] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfdcb91b9fb942f591d2cccaa6bc8b25 (127.8.14.66:36701), 9a9e390c492844e6912aa8fd56a018b5 (127.8.14.67:44493)
I20260808 17:00:57.239940 2125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.236162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:58231 (local address 127.8.14.67:44493)
0808 17:00:57.236214 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:57.236216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:57.236276 (+ 60us) server_negotiation.cc:407] Connection header received
0808 17:00:57.236316 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:57.236317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:57.236328 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:57.236383 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:00:57.236549 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.237306 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.238955 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.239563 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.239654 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.239668 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:57.239824 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 17:00:57.239864 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 17:00:57.239940 2122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.235935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:58079 (local address 127.8.14.66:36701)
0808 17:00:57.235983 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:57.235985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:57.236110 (+ 125us) server_negotiation.cc:407] Connection header received
0808 17:00:57.236204 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:57.236205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:57.236219 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:57.236268 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:00:57.236502 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.237277 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.238955 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.239563 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.239665 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.239675 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:57.239824 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 17:00:57.239864 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:00:57.239986 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.236032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:44493 (local address 127.8.14.65:58231)
0808 17:00:57.236168 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:57.236178 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:57.236269 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:57.236398 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:57.236399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:57.236413 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:00:57.236477 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.236479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.237376 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.237378 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.238846 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.238848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.239654 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.239655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.239734 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.239742 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 17:00:57.239804 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 17:00:57.239862 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":50,"thread_start_us":73,"threads_started":1}
I20260808 17:00:57.239986 2141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.235873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:36701 (local address 127.8.14.65:58079)
0808 17:00:57.236012 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:57.236024 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:57.236128 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:57.236301 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:57.236302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:57.236304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:00:57.236419 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.236421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.237378 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.237378 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.238837 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.238842 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.239666 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.239667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.239734 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.239743 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 17:00:57.239804 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 17:00:57.239860 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":110,"thread_start_us":78,"threads_started":1}
I20260808 17:00:57.240432 2041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b25664e58680497090817eced978d97f" candidate_uuid: "98bd056da6de4d28ad38e94459a232de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a9e390c492844e6912aa8fd56a018b5" is_pre_election: true
I20260808 17:00:57.240581 1855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b25664e58680497090817eced978d97f" candidate_uuid: "98bd056da6de4d28ad38e94459a232de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfdcb91b9fb942f591d2cccaa6bc8b25" is_pre_election: true
I20260808 17:00:57.240644 2041 raft_consensus.cc:2454] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98bd056da6de4d28ad38e94459a232de in term 0.
I20260808 17:00:57.240759 1855 raft_consensus.cc:2454] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98bd056da6de4d28ad38e94459a232de in term 0.
I20260808 17:00:57.240911 1594 leader_election.cc:302] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98bd056da6de4d28ad38e94459a232de, 9a9e390c492844e6912aa8fd56a018b5; no voters:
I20260808 17:00:57.241078 2138 raft_consensus.cc:2781] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:00:57.241165 2138 raft_consensus.cc:482] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:00:57.241184 2138 raft_consensus.cc:3037] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:00:57.242009 2138 raft_consensus.cc:504] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfdcb91b9fb942f591d2cccaa6bc8b25" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 36701 } } peers { permanent_uuid: "98bd056da6de4d28ad38e94459a232de" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 34849 } } peers { permanent_uuid: "9a9e390c492844e6912aa8fd56a018b5" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44493 } }
I20260808 17:00:57.242141 2138 leader_election.cc:288] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [CANDIDATE]: Term 1 election: Requested vote from peers dfdcb91b9fb942f591d2cccaa6bc8b25 (127.8.14.66:36701), 9a9e390c492844e6912aa8fd56a018b5 (127.8.14.67:44493)
I20260808 17:00:57.242347 2041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b25664e58680497090817eced978d97f" candidate_uuid: "98bd056da6de4d28ad38e94459a232de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a9e390c492844e6912aa8fd56a018b5"
I20260808 17:00:57.242347 1855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b25664e58680497090817eced978d97f" candidate_uuid: "98bd056da6de4d28ad38e94459a232de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfdcb91b9fb942f591d2cccaa6bc8b25"
I20260808 17:00:57.242437 1855 raft_consensus.cc:3037] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:00:57.242437 2041 raft_consensus.cc:3037] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:00:57.243479 2041 raft_consensus.cc:2454] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98bd056da6de4d28ad38e94459a232de in term 1.
I20260808 17:00:57.243479 1855 raft_consensus.cc:2454] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98bd056da6de4d28ad38e94459a232de in term 1.
I20260808 17:00:57.243673 1594 leader_election.cc:302] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98bd056da6de4d28ad38e94459a232de, 9a9e390c492844e6912aa8fd56a018b5; no voters:
I20260808 17:00:57.243777 2138 raft_consensus.cc:2781] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:00:57.243942 2138 raft_consensus.cc:686] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [term 1 LEADER]: Becoming Leader. State: Replica: 98bd056da6de4d28ad38e94459a232de, State: Running, Role: LEADER
I20260808 17:00:57.244066 2138 consensus_queue.cc:237] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfdcb91b9fb942f591d2cccaa6bc8b25" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 36701 } } peers { permanent_uuid: "98bd056da6de4d28ad38e94459a232de" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 34849 } } peers { permanent_uuid: "9a9e390c492844e6912aa8fd56a018b5" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44493 } }
I20260808 17:00:57.244963 1484 catalog_manager.cc:5495] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de reported cstate change: term changed from 0 to 1, leader changed from <none> to 98bd056da6de4d28ad38e94459a232de (127.8.14.65). New cstate: current_term: 1 leader_uuid: "98bd056da6de4d28ad38e94459a232de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfdcb91b9fb942f591d2cccaa6bc8b25" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 36701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98bd056da6de4d28ad38e94459a232de" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 34849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a9e390c492844e6912aa8fd56a018b5" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44493 } health_report { overall_health: UNKNOWN } } }
W20260808 17:00:57.270187 2110 tablet.cc:2369] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:00:57.282709 2126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.280196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36448 (local address 127.8.14.65:34849)
0808 17:00:57.280253 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:57.280255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:57.280263 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:00:57.280324 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:57.280325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:57.280347 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:57.280436 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:00:57.280599 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.281113 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.281905 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.282129 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.282209 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.282379 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:00:57.282487 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:00:57.282584 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:57.282603 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 17:00:57.282622 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 17:00:57.284614 2041 raft_consensus.cc:1261] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5 [term 1 FOLLOWER]: Refusing update from remote peer 98bd056da6de4d28ad38e94459a232de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:00:57.284615 1855 raft_consensus.cc:1261] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25 [term 1 FOLLOWER]: Refusing update from remote peer 98bd056da6de4d28ad38e94459a232de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:00:57.284986 2145 consensus_queue.cc:1035] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfdcb91b9fb942f591d2cccaa6bc8b25" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 36701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:00:57.285125 2138 consensus_queue.cc:1035] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a9e390c492844e6912aa8fd56a018b5" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 44493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:00:57.287945 2150 mvcc.cc:205] Tried to move back new op lower bound from 7316309841034952704 to 7316309840872357888. Current Snapshot: MvccSnapshot[applied={T|T < 7316309841034952704}]
I20260808 17:00:57.288081 2148 mvcc.cc:205] Tried to move back new op lower bound from 7316309841034952704 to 7316309840872357888. Current Snapshot: MvccSnapshot[applied={T|T < 7316309841034952704 or (T in {7316309841034952704})}]
I20260808 17:00:57.290045 1483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:00:57.290148 1483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:00:57.290958 1483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ccda69589d23446786b5f3f4123f22af: Sending DeleteTablet for 3 replicas of tablet b25664e58680497090817eced978d97f
I20260808 17:00:57.291277 1649 tablet_service.cc:1514] Processing DeleteTablet for tablet b25664e58680497090817eced978d97f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:00:57 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:36434
I20260808 17:00:57.291298 2015 tablet_service.cc:1514] Processing DeleteTablet for tablet b25664e58680497090817eced978d97f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:00:57 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:38988
I20260808 17:00:57.291543 2159 tablet_replica.cc:330] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5: stopping tablet replica
I20260808 17:00:57.291580 1829 tablet_service.cc:1514] Processing DeleteTablet for tablet b25664e58680497090817eced978d97f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:00:57 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:43900
I20260808 17:00:57.291644 2159 raft_consensus.cc:2229] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:00:57.291745 2159 raft_consensus.cc:2258] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:00:57.291844 2160 tablet_replica.cc:330] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25: stopping tablet replica
I20260808 17:00:57.291879 2158 tablet_replica.cc:330] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de: stopping tablet replica
I20260808 17:00:57.291971 2158 raft_consensus.cc:2229] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:00:57.291996 2160 raft_consensus.cc:2229] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:00:57.292065 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 1567
I20260808 17:00:57.292073 2158 raft_consensus.cc:2258] T b25664e58680497090817eced978d97f P 98bd056da6de4d28ad38e94459a232de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:00:57.294698 2160 raft_consensus.cc:2258] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:00:57.297144 2160 ts_tablet_manager.cc:1905] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:00:57.298604 2160 ts_tablet_manager.cc:1918] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:00:57.298696 2160 log.cc:1196] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/b25664e58680497090817eced978d97f
I20260808 17:00:57.299064 2160 ts_tablet_manager.cc:1939] T b25664e58680497090817eced978d97f P dfdcb91b9fb942f591d2cccaa6bc8b25: Deleting consensus metadata
I20260808 17:00:57.299460 1459 catalog_manager.cc:4841] TS dfdcb91b9fb942f591d2cccaa6bc8b25 (127.8.14.66:36701): tablet b25664e58680497090817eced978d97f (table test-table [id=384303ddb77d4bbd8ff2b0ee812a4955]) successfully deleted
I20260808 17:00:57.299479 2159 ts_tablet_manager.cc:1905] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:00:57.301462 2159 ts_tablet_manager.cc:1918] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:00:57.301576 2159 log.cc:1196] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/b25664e58680497090817eced978d97f
W20260808 17:00:57.301671 1462 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.14.65:34849 (error 108)
I20260808 17:00:57.301906 2159 ts_tablet_manager.cc:1939] T b25664e58680497090817eced978d97f P 9a9e390c492844e6912aa8fd56a018b5: Deleting consensus metadata
I20260808 17:00:57.301975 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 1745
I20260808 17:00:57.302146 2119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.302019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:34849 (local address 127.0.0.1:36464)
0808 17:00:57.302065 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:57.302098 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.65:34849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 17:00:57.302346 1459 catalog_manager.cc:4841] TS 9a9e390c492844e6912aa8fd56a018b5 (127.8.14.67:44493): tablet b25664e58680497090817eced978d97f (table test-table [id=384303ddb77d4bbd8ff2b0ee812a4955]) successfully deleted
W20260808 17:00:57.302487 1462 catalog_manager.cc:4584] TS 98bd056da6de4d28ad38e94459a232de (127.8.14.65:34849): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b25664e58680497090817eced978d97f: Network error: Client connection negotiation failed: client connection to 127.8.14.65:34849: connect: Connection refused (error 111)
I20260808 17:00:57.310828 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 1924
I20260808 17:00:57.321425 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 1437
2026-08-08T17:00:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2267 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T17:00:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:00:57Z Disabled control of system clock
I20260808 17:00:57.353292 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:46477
--webserver_interface=127.8.14.126
--webserver_port=0
--builtin_ntp_servers=127.8.14.84:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:46477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:00:57.433913 2171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:57.434110 2171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:57.434130 2171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:57.435194 2171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:00:57.435226 2171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:57.435240 2171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:57.435251 2171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:00:57.435263 2171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:00:57.436518 2171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:46477
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:46477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:57.436702 2171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:57.436926 2171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:57.441582 2183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:57.441618 2181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:57.441627 2180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:57.441890 2171 server_base.cc:1034] running on GCE node
I20260808 17:00:57.442134 2171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:57.442405 2171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:57.443564 2171 hybrid_clock.cc:648] HybridClock initialized: now 1786208457443548 us; error 46 us; skew 500 ppm
I20260808 17:00:57.444857 2171 webserver.cc:460] Webserver started at http://127.8.14.126:36051/ using document root <none> and password file <none>
I20260808 17:00:57.445106 2171 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:57.445178 2171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:57.445338 2171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:00:57.446333 2171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "63cd869c68c34296affd2f48a5f26859"
format_stamp: "Formatted at 2026-08-08 17:00:57 on dist-test-slave-1nl6"
server_key: "2d646eeb851f6c854949cfbcaba6e4ce"
server_key_iv: "879353f3de70e79aedddaa681f1d4279"
server_key_version: "encryptionkey@0"
I20260808 17:00:57.446695 2171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "63cd869c68c34296affd2f48a5f26859"
format_stamp: "Formatted at 2026-08-08 17:00:57 on dist-test-slave-1nl6"
server_key: "2d646eeb851f6c854949cfbcaba6e4ce"
server_key_iv: "879353f3de70e79aedddaa681f1d4279"
server_key_version: "encryptionkey@0"
I20260808 17:00:57.448130 2171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:00:57.448854 2190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:57.449028 2171 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:00:57.449128 2171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "63cd869c68c34296affd2f48a5f26859"
format_stamp: "Formatted at 2026-08-08 17:00:57 on dist-test-slave-1nl6"
server_key: "2d646eeb851f6c854949cfbcaba6e4ce"
server_key_iv: "879353f3de70e79aedddaa681f1d4279"
server_key_version: "encryptionkey@0"
I20260808 17:00:57.449213 2171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:57.464912 2171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:57.465191 2171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:57.465342 2171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:57.469424 2171 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:46477
I20260808 17:00:57.469460 2256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:46477 every 8 connection(s)
I20260808 17:00:57.469769 2171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:00:57.470729 2257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:00:57.472914 2257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859: Bootstrap starting.
I20260808 17:00:57.473544 2257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859: Neither blocks nor log segments found. Creating new log.
I20260808 17:00:57.473856 2257 log.cc:824] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859: Log is configured to *not* fsync() on all Append() calls
I20260808 17:00:57.474509 2257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859: No bootstrap required, opened a new log
I20260808 17:00:57.476120 2257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cd869c68c34296affd2f48a5f26859" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46477 } }
I20260808 17:00:57.476251 2257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:00:57.476298 2257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63cd869c68c34296affd2f48a5f26859, State: Initialized, Role: FOLLOWER
I20260808 17:00:57.476467 2257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cd869c68c34296affd2f48a5f26859" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46477 } }
I20260808 17:00:57.476552 2257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:00:57.476603 2257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:00:57.476656 2257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:00:57.477224 2257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cd869c68c34296affd2f48a5f26859" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46477 } }
I20260808 17:00:57.477334 2257 leader_election.cc:302] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63cd869c68c34296affd2f48a5f26859; no voters:
I20260808 17:00:57.477497 2257 leader_election.cc:288] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:00:57.477577 2262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:00:57.477764 2257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:00:57.477756 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 2171
I20260808 17:00:57.477782 2262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [term 1 LEADER]: Becoming Leader. State: Replica: 63cd869c68c34296affd2f48a5f26859, State: Running, Role: LEADER
I20260808 17:00:57.477870 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:00:57.478008 2262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cd869c68c34296affd2f48a5f26859" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46477 } }
I20260808 17:00:57.478080 8249 external_mini_cluster.cc:1428] Setting key 074e44c1af3546af6363e596818ccee4
I20260808 17:00:57.478541 2264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63cd869c68c34296affd2f48a5f26859" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cd869c68c34296affd2f48a5f26859" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46477 } } }
I20260808 17:00:57.478631 2264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [sys.catalog]: This master's current role is: LEADER
I20260808 17:00:57.479174 2265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63cd869c68c34296affd2f48a5f26859. Latest consensus state: current_term: 1 leader_uuid: "63cd869c68c34296affd2f48a5f26859" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cd869c68c34296affd2f48a5f26859" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 46477 } } }
I20260808 17:00:57.479259 2265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859 [sys.catalog]: This master's current role is: LEADER
W20260808 17:00:57.479728 2286 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:00:57.479823 2286 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:00:57.480072 2277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:00:57.480307 2277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:00:57.481897 2277 catalog_manager.cc:1347] Generated new cluster ID: e6abaea1de9343ffbf69ac0efa569abc
I20260808 17:00:57.481952 2277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:00:57.484334 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.478743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47532 (local address 127.8.14.126:46477)
0808 17:00:57.478978 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:57.478988 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:57.479593 (+ 605us) server_negotiation.cc:407] Connection header received
0808 17:00:57.479744 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:57.479769 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:57.479845 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:57.479986 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:57.480187 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.481322 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.482380 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.482792 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.482943 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.483685 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:57.483689 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:57.483690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:00:57.483877 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:57.483910 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:00:57.483921 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:57.484002 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:57.484078 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 17:00:57.484161 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":105,"threads_started":1}
I20260808 17:00:57.487720 2277 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:00:57.488204 2277 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:00:57.505883 2277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859: Generated new TSK 0
I20260808 17:00:57.506115 2277 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:00:57.512511 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46477
--builtin_ntp_servers=127.8.14.84:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:00:57.609606 2291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:57.609820 2291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:57.609858 2291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:57.611033 2291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:57.611088 2291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:57.611129 2291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 17:00:57.612474 2291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:57.612735 2291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:57.613031 2291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:57.619048 2305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:57.619114 2302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:57.619387 2297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:57.619478 2291 server_base.cc:1034] running on GCE node
I20260808 17:00:57.619709 2291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:57.619999 2291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:57.621186 2291 hybrid_clock.cc:648] HybridClock initialized: now 1786208457621180 us; error 35 us; skew 500 ppm
I20260808 17:00:57.622610 2291 webserver.cc:460] Webserver started at http://127.8.14.65:40747/ using document root <none> and password file <none>
I20260808 17:00:57.622936 2291 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:57.623020 2291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:57.623121 2291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:00:57.624140 2291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "30ba8ac122354ff684104423f5b9dfdd"
format_stamp: "Formatted at 2026-08-08 17:00:57 on dist-test-slave-1nl6"
server_key: "bf4a9f434c65dfa9c77efb697908d82e"
server_key_iv: "e259fbc39e4eea6ab12884ed63480044"
server_key_version: "encryptionkey@0"
I20260808 17:00:57.624439 2291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "30ba8ac122354ff684104423f5b9dfdd"
format_stamp: "Formatted at 2026-08-08 17:00:57 on dist-test-slave-1nl6"
server_key: "bf4a9f434c65dfa9c77efb697908d82e"
server_key_iv: "e259fbc39e4eea6ab12884ed63480044"
server_key_version: "encryptionkey@0"
I20260808 17:00:57.625694 2291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:00:57.626355 2313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:57.626511 2291 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:00:57.626592 2291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "30ba8ac122354ff684104423f5b9dfdd"
format_stamp: "Formatted at 2026-08-08 17:00:57 on dist-test-slave-1nl6"
server_key: "bf4a9f434c65dfa9c77efb697908d82e"
server_key_iv: "e259fbc39e4eea6ab12884ed63480044"
server_key_version: "encryptionkey@0"
I20260808 17:00:57.626646 2291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:57.644500 2291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:57.644743 2291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:57.644838 2291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:57.645009 2291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:00:57.645354 2291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:00:57.645390 2291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:57.645411 2291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:00:57.645434 2291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:57.651113 2291 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:43853
I20260808 17:00:57.651168 2480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:43853 every 8 connection(s)
I20260808 17:00:57.651494 2291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 17:00:57.656538 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.652216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:59471 (local address 127.8.14.126:46477)
0808 17:00:57.652285 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:57.652287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:57.652546 (+ 259us) server_negotiation.cc:407] Connection header received
0808 17:00:57.653144 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:57.653145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:57.653157 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:57.653216 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:57.653865 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.654352 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.655336 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.655514 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.655566 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.656304 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:57.656305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:57.656305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:00:57.656309 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:57.656323 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:00:57.656327 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:57.656371 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:57.656456 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 17:00:57.656480 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 17:00:57.656692 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.652226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46477 (local address 127.8.14.65:59471)
0808 17:00:57.652429 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:57.652460 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:57.653021 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:57.653293 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:57.653296 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:57.653304 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:00:57.653779 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.653783 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.654431 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.654432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.655257 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.655259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.655588 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.655589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.655691 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.656170 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:00:57.656170 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:00:57.656196 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:00:57.656218 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:00:57.656382 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:00:57.656386 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:00:57.656388 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:00:57.656439 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 17:00:57.656512 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":106,"threads_started":1}
I20260808 17:00:57.657202 2482 heartbeater.cc:347] Connected to a master server at 127.8.14.126:46477
I20260808 17:00:57.657310 2482 heartbeater.cc:464] Registering TS with master...
I20260808 17:00:57.657483 2482 heartbeater.cc:511] Master 127.8.14.126:46477 requested a full tablet report, sending...
I20260808 17:00:57.657963 2211 ts_manager.cc:194] Registered new tserver with Master: 30ba8ac122354ff684104423f5b9dfdd (127.8.14.65:43853)
I20260808 17:00:57.658533 2211 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.14.65:59471
I20260808 17:00:57.658947 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 2291
I20260808 17:00:57.659054 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 17:00:57.659256 8249 external_mini_cluster.cc:1428] Setting key 9560b569664ff583ed54d1435322f204
I20260808 17:00:57.660140 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46477
--builtin_ntp_servers=127.8.14.84:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:00:57.740674 2486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:57.740908 2486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:57.740944 2486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:57.742053 2486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:57.742122 2486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:57.742162 2486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 17:00:57.743592 2486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:57.743871 2486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:57.744254 2486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:57.749420 2495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:57.749428 2493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:57.749627 2486 server_base.cc:1034] running on GCE node
W20260808 17:00:57.749429 2499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:57.749892 2486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:57.750168 2486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:57.751348 2486 hybrid_clock.cc:648] HybridClock initialized: now 1786208457751323 us; error 53 us; skew 500 ppm
I20260808 17:00:57.752607 2486 webserver.cc:460] Webserver started at http://127.8.14.66:43941/ using document root <none> and password file <none>
I20260808 17:00:57.752859 2486 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:57.752931 2486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:57.753089 2486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:00:57.754112 2486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "4d56da4c857e4e8ea564172c26a26744"
format_stamp: "Formatted at 2026-08-08 17:00:57 on dist-test-slave-1nl6"
server_key: "2c7ab85348562d1b45895d23e1a2ff99"
server_key_iv: "d70bfbc2665eabe1ea4e68f70c615f63"
server_key_version: "encryptionkey@0"
I20260808 17:00:57.754483 2486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "4d56da4c857e4e8ea564172c26a26744"
format_stamp: "Formatted at 2026-08-08 17:00:57 on dist-test-slave-1nl6"
server_key: "2c7ab85348562d1b45895d23e1a2ff99"
server_key_iv: "d70bfbc2665eabe1ea4e68f70c615f63"
server_key_version: "encryptionkey@0"
I20260808 17:00:57.755877 2486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:00:57.756587 2505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:57.756752 2486 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:00:57.756858 2486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "4d56da4c857e4e8ea564172c26a26744"
format_stamp: "Formatted at 2026-08-08 17:00:57 on dist-test-slave-1nl6"
server_key: "2c7ab85348562d1b45895d23e1a2ff99"
server_key_iv: "d70bfbc2665eabe1ea4e68f70c615f63"
server_key_version: "encryptionkey@0"
I20260808 17:00:57.756950 2486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:57.777088 2486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:57.777370 2486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:57.777515 2486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:57.777783 2486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:00:57.778172 2486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:00:57.778227 2486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:57.778265 2486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:00:57.778293 2486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:57.783993 2486 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:32879
I20260808 17:00:57.784027 2667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:32879 every 8 connection(s)
I20260808 17:00:57.784354 2486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 17:00:57.785532 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 2486
I20260808 17:00:57.785640 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 17:00:57.785851 8249 external_mini_cluster.cc:1428] Setting key 06509279627c07316fa37709cb88d5b3
I20260808 17:00:57.786891 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46477
--builtin_ntp_servers=127.8.14.84:37349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 17:00:57.790464 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.785046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:58817 (local address 127.8.14.126:46477)
0808 17:00:57.785093 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:57.785096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:57.785326 (+ 230us) server_negotiation.cc:407] Connection header received
0808 17:00:57.786024 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:57.786026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:57.786045 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:57.786124 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:57.787040 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.787588 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.788971 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.789281 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.789367 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.790129 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:57.790131 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:57.790131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:00:57.790136 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:57.790155 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:00:57.790159 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:57.790218 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:57.790307 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 17:00:57.790346 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:00:57.790501 2672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.785045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46477 (local address 127.8.14.66:58817)
0808 17:00:57.785205 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:57.785237 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:57.785901 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:57.786211 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:57.786216 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:57.786227 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:00:57.786903 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.786937 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.787686 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.787687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.788877 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.788880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.789371 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.789371 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.789474 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.789989 (+ 515us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:00:57.789990 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:00:57.790015 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:00:57.790035 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:00:57.790224 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:00:57.790228 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:00:57.790231 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:00:57.790281 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 17:00:57.790359 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260808 17:00:57.791004 2669 heartbeater.cc:347] Connected to a master server at 127.8.14.126:46477
I20260808 17:00:57.791108 2669 heartbeater.cc:464] Registering TS with master...
I20260808 17:00:57.791285 2669 heartbeater.cc:511] Master 127.8.14.126:46477 requested a full tablet report, sending...
I20260808 17:00:57.791671 2211 ts_manager.cc:194] Registered new tserver with Master: 4d56da4c857e4e8ea564172c26a26744 (127.8.14.66:32879)
I20260808 17:00:57.792220 2211 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.14.66:58817
W20260808 17:00:57.870043 2673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:57.870234 2673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:57.870254 2673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:57.871405 2673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:57.871445 2673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:57.871472 2673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 17:00:57.872844 2673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:37349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:46477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:57.873024 2673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:57.873258 2673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:57.878328 2682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:57.878316 2678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:57.878399 2679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:57.878504 2673 server_base.cc:1034] running on GCE node
I20260808 17:00:57.878759 2673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:57.879053 2673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:57.880215 2673 hybrid_clock.cc:648] HybridClock initialized: now 1786208457880200 us; error 33 us; skew 500 ppm
I20260808 17:00:57.881422 2673 webserver.cc:460] Webserver started at http://127.8.14.67:34801/ using document root <none> and password file <none>
I20260808 17:00:57.881662 2673 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:57.881732 2673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:57.881896 2673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:00:57.883019 2673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "f07469e81b09421b85efe529957b07d0"
format_stamp: "Formatted at 2026-08-08 17:00:57 on dist-test-slave-1nl6"
server_key: "037ae3a5328774f824b74fcd2cf566ce"
server_key_iv: "d3491ddf7fe9e963c67bb20a84014622"
server_key_version: "encryptionkey@0"
I20260808 17:00:57.883396 2673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "f07469e81b09421b85efe529957b07d0"
format_stamp: "Formatted at 2026-08-08 17:00:57 on dist-test-slave-1nl6"
server_key: "037ae3a5328774f824b74fcd2cf566ce"
server_key_iv: "d3491ddf7fe9e963c67bb20a84014622"
server_key_version: "encryptionkey@0"
I20260808 17:00:57.884723 2673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:00:57.885452 2691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:57.885632 2673 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:00:57.885737 2673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "f07469e81b09421b85efe529957b07d0"
format_stamp: "Formatted at 2026-08-08 17:00:57 on dist-test-slave-1nl6"
server_key: "037ae3a5328774f824b74fcd2cf566ce"
server_key_iv: "d3491ddf7fe9e963c67bb20a84014622"
server_key_version: "encryptionkey@0"
I20260808 17:00:57.885816 2673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:57.900455 2673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:57.900727 2673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:57.900872 2673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:57.901087 2673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:00:57.901410 2673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:00:57.901473 2673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:57.901516 2673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:00:57.901546 2673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:57.907217 2673 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:40689
I20260808 17:00:57.907262 2860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:40689 every 8 connection(s)
I20260808 17:00:57.907624 2673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 17:00:57.912442 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.908340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:45289 (local address 127.8.14.126:46477)
0808 17:00:57.908421 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:57.908423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:57.908612 (+ 189us) server_negotiation.cc:407] Connection header received
0808 17:00:57.909068 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:57.909069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:57.909079 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:57.909132 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:57.909733 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.910225 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.911207 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.911394 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.911448 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.912247 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:57.912248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:57.912248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:00:57.912252 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:57.912263 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:00:57.912266 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:57.912303 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:57.912363 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 17:00:57.912384 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 17:00:57.912628 2864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.908289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:46477 (local address 127.8.14.67:45289)
0808 17:00:57.908488 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:57.908520 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:57.908984 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:57.909181 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:57.909184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:57.909191 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:00:57.909665 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.909669 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.910300 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.910301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.911135 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.911138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.911468 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.911469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.911573 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.912125 (+ 552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:00:57.912126 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:00:57.912153 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:00:57.912175 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:00:57.912305 (+ 130us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:00:57.912307 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:00:57.912309 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:00:57.912356 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 17:00:57.912428 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
I20260808 17:00:57.913178 2861 heartbeater.cc:347] Connected to a master server at 127.8.14.126:46477
I20260808 17:00:57.913209 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 2673
I20260808 17:00:57.913283 2861 heartbeater.cc:464] Registering TS with master...
I20260808 17:00:57.913311 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 17:00:57.913496 2861 heartbeater.cc:511] Master 127.8.14.126:46477 requested a full tablet report, sending...
I20260808 17:00:57.913517 8249 external_mini_cluster.cc:1428] Setting key 2950c98f18ad5ed20e9d65e706df4ce4
I20260808 17:00:57.913976 2210 ts_manager.cc:194] Registered new tserver with Master: f07469e81b09421b85efe529957b07d0 (127.8.14.67:40689)
I20260808 17:00:57.914471 2210 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.14.67:45289
I20260808 17:00:57.914544 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:00:57.918627 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.916095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47548 (local address 127.8.14.126:46477)
0808 17:00:57.916165 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:57.916167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:57.916221 (+ 54us) server_negotiation.cc:407] Connection header received
0808 17:00:57.916262 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:57.916263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:57.916271 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:57.916323 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:57.916459 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.916891 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.917578 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.917746 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.917793 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.918432 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:57.918433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:57.918433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:00:57.918437 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:57.918446 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:00:57.918448 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:57.918485 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:57.918523 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 17:00:57.918538 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 17:00:57.920457 2210 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:00:57.921972 2210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:00:57.928360 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.924277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:40689 (local address 127.0.0.1:59098)
0808 17:00:57.924460 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:57.924501 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:57.924569 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:57.924916 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:57.924919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:57.924921 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:00:57.924988 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.924990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.926258 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.926260 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.927348 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.927351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.928038 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.928039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.928115 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.928128 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 17:00:57.928202 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 17:00:57.928273 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":3,"thread_start_us":105,"threads_started":1}
I20260808 17:00:57.928360 2876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.924353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59098 (local address 127.8.14.67:40689)
0808 17:00:57.924703 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:57.924709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:57.924718 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:00:57.924753 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:57.924755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:57.924771 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:57.924903 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:00:57.925076 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.926171 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.927442 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.927945 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.928006 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.928113 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:57.928233 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 17:00:57.928274 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":76,"threads_started":1}
I20260808 17:00:57.928386 2879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.924610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58358 (local address 127.8.14.65:43853)
0808 17:00:57.924817 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:57.924822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:57.924832 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:00:57.925136 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:57.925138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:57.925155 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:57.925235 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:00:57.925374 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.926393 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.927479 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.927954 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.928048 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.928135 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:57.928258 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 17:00:57.928305 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":96,"threads_started":1}
I20260808 17:00:57.928612 2878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.924508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:43853 (local address 127.0.0.1:58358)
0808 17:00:57.924752 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:57.924761 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:57.924805 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:57.925251 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:57.925252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:57.925254 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:00:57.925288 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.925289 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.926475 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.926476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.927385 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.927387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.928066 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.928067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.928130 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.928136 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 17:00:57.928212 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 17:00:57.928544 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":280,"thread_start_us":70,"threads_started":1}
I20260808 17:00:57.929266 2874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.924282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:32879 (local address 127.0.0.1:51606)
0808 17:00:57.924590 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:57.924600 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:57.924648 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:57.925009 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:57.925010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:57.925012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:00:57.925060 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.925062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.926026 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.926027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.927352 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:57.927354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.929042 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.929043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:57.929126 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.929130 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:00:57.929162 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 17:00:57.929201 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":3,"thread_start_us":69,"threads_started":1}
I20260808 17:00:57.929348 2389 tablet_service.cc:1467] Processing CreateTablet for tablet 3f62cf083ad54103aa600074596b9381 (DEFAULT_TABLE table=test-table [id=2754269b18be4135aa855eccb0bd22dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:00:57.929491 2766 tablet_service.cc:1467] Processing CreateTablet for tablet 3f62cf083ad54103aa600074596b9381 (DEFAULT_TABLE table=test-table [id=2754269b18be4135aa855eccb0bd22dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:00:57.929631 2877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:57.924409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51606 (local address 127.8.14.66:32879)
0808 17:00:57.924831 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:57.924835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:57.924844 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:00:57.924887 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:57.924889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:57.924903 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:57.925005 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:00:57.925144 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.925931 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.928155 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:57.928940 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:57.929392 (+ 452us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:57.929484 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:57.929513 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 17:00:57.929554 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":82,"threads_started":1}
I20260808 17:00:57.929765 2389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f62cf083ad54103aa600074596b9381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:00:57.929872 2766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f62cf083ad54103aa600074596b9381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:00:57.932466 2880 tablet_bootstrap.cc:492] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd: Bootstrap starting.
I20260808 17:00:57.932642 2881 tablet_bootstrap.cc:492] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0: Bootstrap starting.
I20260808 17:00:57.933164 2880 tablet_bootstrap.cc:654] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd: Neither blocks nor log segments found. Creating new log.
I20260808 17:00:57.933303 2881 tablet_bootstrap.cc:654] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0: Neither blocks nor log segments found. Creating new log.
I20260808 17:00:57.933250 2572 tablet_service.cc:1467] Processing CreateTablet for tablet 3f62cf083ad54103aa600074596b9381 (DEFAULT_TABLE table=test-table [id=2754269b18be4135aa855eccb0bd22dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:00:57.933535 2880 log.cc:824] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd: Log is configured to *not* fsync() on all Append() calls
I20260808 17:00:57.933609 2572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f62cf083ad54103aa600074596b9381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:00:57.933629 2881 log.cc:824] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0: Log is configured to *not* fsync() on all Append() calls
I20260808 17:00:57.934324 2881 tablet_bootstrap.cc:492] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0: No bootstrap required, opened a new log
I20260808 17:00:57.934370 2880 tablet_bootstrap.cc:492] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd: No bootstrap required, opened a new log
I20260808 17:00:57.934412 2881 ts_tablet_manager.cc:1397] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:00:57.934429 2880 ts_tablet_manager.cc:1397] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:00:57.936290 2881 raft_consensus.cc:348] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07469e81b09421b85efe529957b07d0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40689 } } peers { permanent_uuid: "4d56da4c857e4e8ea564172c26a26744" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 32879 } } peers { permanent_uuid: "30ba8ac122354ff684104423f5b9dfdd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43853 } }
I20260808 17:00:57.936450 2881 raft_consensus.cc:374] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:00:57.936496 2881 raft_consensus.cc:729] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f07469e81b09421b85efe529957b07d0, State: Initialized, Role: FOLLOWER
I20260808 17:00:57.936628 2881 consensus_queue.cc:260] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07469e81b09421b85efe529957b07d0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40689 } } peers { permanent_uuid: "4d56da4c857e4e8ea564172c26a26744" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 32879 } } peers { permanent_uuid: "30ba8ac122354ff684104423f5b9dfdd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43853 } }
I20260808 17:00:57.936921 2881 ts_tablet_manager.cc:1428] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:00:57.936942 2861 heartbeater.cc:503] Master 127.8.14.126:46477 was elected leader, sending a full tablet report...
I20260808 17:00:57.937091 2880 raft_consensus.cc:348] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07469e81b09421b85efe529957b07d0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40689 } } peers { permanent_uuid: "4d56da4c857e4e8ea564172c26a26744" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 32879 } } peers { permanent_uuid: "30ba8ac122354ff684104423f5b9dfdd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43853 } }
I20260808 17:00:57.937263 2880 raft_consensus.cc:374] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:00:57.937278 2887 tablet_bootstrap.cc:492] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744: Bootstrap starting.
I20260808 17:00:57.937307 2880 raft_consensus.cc:729] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ba8ac122354ff684104423f5b9dfdd, State: Initialized, Role: FOLLOWER
I20260808 17:00:57.937426 2880 consensus_queue.cc:260] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07469e81b09421b85efe529957b07d0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40689 } } peers { permanent_uuid: "4d56da4c857e4e8ea564172c26a26744" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 32879 } } peers { permanent_uuid: "30ba8ac122354ff684104423f5b9dfdd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43853 } }
I20260808 17:00:57.937711 2880 ts_tablet_manager.cc:1428] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 17:00:57.937758 2482 heartbeater.cc:503] Master 127.8.14.126:46477 was elected leader, sending a full tablet report...
I20260808 17:00:57.937937 2887 tablet_bootstrap.cc:654] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744: Neither blocks nor log segments found. Creating new log.
I20260808 17:00:57.938272 2887 log.cc:824] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744: Log is configured to *not* fsync() on all Append() calls
I20260808 17:00:57.939117 2887 tablet_bootstrap.cc:492] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744: No bootstrap required, opened a new log
I20260808 17:00:57.939172 2887 ts_tablet_manager.cc:1397] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:00:57.940953 2887 raft_consensus.cc:348] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07469e81b09421b85efe529957b07d0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40689 } } peers { permanent_uuid: "4d56da4c857e4e8ea564172c26a26744" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 32879 } } peers { permanent_uuid: "30ba8ac122354ff684104423f5b9dfdd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43853 } }
I20260808 17:00:57.941074 2887 raft_consensus.cc:374] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:00:57.941095 2887 raft_consensus.cc:729] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d56da4c857e4e8ea564172c26a26744, State: Initialized, Role: FOLLOWER
I20260808 17:00:57.941233 2887 consensus_queue.cc:260] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07469e81b09421b85efe529957b07d0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40689 } } peers { permanent_uuid: "4d56da4c857e4e8ea564172c26a26744" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 32879 } } peers { permanent_uuid: "30ba8ac122354ff684104423f5b9dfdd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43853 } }
I20260808 17:00:57.941502 2887 ts_tablet_manager.cc:1428] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:00:57.941525 2669 heartbeater.cc:503] Master 127.8.14.126:46477 was elected leader, sending a full tablet report...
W20260808 17:00:58.035044 2671 tablet.cc:2369] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:00:58.054942 2888 raft_consensus.cc:482] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:00:58.055110 2888 raft_consensus.cc:504] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07469e81b09421b85efe529957b07d0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40689 } } peers { permanent_uuid: "4d56da4c857e4e8ea564172c26a26744" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 32879 } } peers { permanent_uuid: "30ba8ac122354ff684104423f5b9dfdd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43853 } }
I20260808 17:00:58.055549 2888 leader_election.cc:288] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f07469e81b09421b85efe529957b07d0 (127.8.14.67:40689), 4d56da4c857e4e8ea564172c26a26744 (127.8.14.66:32879)
I20260808 17:00:58.059504 2876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:58.055813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:40059 (local address 127.8.14.67:40689)
0808 17:00:58.055877 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:58.055881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:58.055893 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:00:58.055929 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:58.055931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:58.055940 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:58.056024 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:00:58.056195 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:58.056966 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:58.058507 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:58.059093 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:58.059172 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:58.059183 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:58.059386 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 17:00:58.059429 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 17:00:58.059504 2877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:58.055963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:60093 (local address 127.8.14.66:32879)
0808 17:00:58.056034 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:58.056036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:58.056067 (+ 31us) server_negotiation.cc:407] Connection header received
0808 17:00:58.056140 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:58.056141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:58.056148 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:58.056225 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:00:58.056371 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:58.057045 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:58.058506 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:58.059093 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:58.059172 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:58.059183 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:58.059386 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 17:00:58.059429 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 17:00:58.059553 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:58.055732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:40689 (local address 127.8.14.65:40059)
0808 17:00:58.055782 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:58.055796 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:58.055849 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:58.056034 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:58.056034 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:58.056036 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:00:58.056115 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:58.056117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:58.057067 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:58.057068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:58.058405 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:58.058408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:58.059188 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:58.059190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:58.059284 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:58.059291 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 17:00:58.059381 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 17:00:58.059418 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":43}
I20260808 17:00:58.059562 2892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:58.055821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:32879 (local address 127.8.14.65:60093)
0808 17:00:58.056006 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:58.056016 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:58.056070 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:58.056248 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:58.056248 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:58.056250 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:00:58.056301 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:58.056302 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:58.057156 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:58.057156 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:58.058405 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:58.058408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:58.059189 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:58.059190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:58.059284 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:58.059291 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 17:00:58.059368 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 17:00:58.059415 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":11,"thread_start_us":119,"threads_started":1}
I20260808 17:00:58.060086 2798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f62cf083ad54103aa600074596b9381" candidate_uuid: "30ba8ac122354ff684104423f5b9dfdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07469e81b09421b85efe529957b07d0" is_pre_election: true
I20260808 17:00:58.060092 2599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f62cf083ad54103aa600074596b9381" candidate_uuid: "30ba8ac122354ff684104423f5b9dfdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d56da4c857e4e8ea564172c26a26744" is_pre_election: true
I20260808 17:00:58.060253 2599 raft_consensus.cc:2454] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30ba8ac122354ff684104423f5b9dfdd in term 0.
I20260808 17:00:58.060272 2798 raft_consensus.cc:2454] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30ba8ac122354ff684104423f5b9dfdd in term 0.
I20260808 17:00:58.060487 2315 leader_election.cc:302] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ba8ac122354ff684104423f5b9dfdd, 4d56da4c857e4e8ea564172c26a26744; no voters:
I20260808 17:00:58.060671 2888 raft_consensus.cc:2781] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:00:58.060755 2888 raft_consensus.cc:482] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:00:58.060788 2888 raft_consensus.cc:3037] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:00:58.061581 2888 raft_consensus.cc:504] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07469e81b09421b85efe529957b07d0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40689 } } peers { permanent_uuid: "4d56da4c857e4e8ea564172c26a26744" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 32879 } } peers { permanent_uuid: "30ba8ac122354ff684104423f5b9dfdd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43853 } }
I20260808 17:00:58.061741 2888 leader_election.cc:288] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [CANDIDATE]: Term 1 election: Requested vote from peers f07469e81b09421b85efe529957b07d0 (127.8.14.67:40689), 4d56da4c857e4e8ea564172c26a26744 (127.8.14.66:32879)
I20260808 17:00:58.061880 2798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f62cf083ad54103aa600074596b9381" candidate_uuid: "30ba8ac122354ff684104423f5b9dfdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07469e81b09421b85efe529957b07d0"
I20260808 17:00:58.061889 2599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f62cf083ad54103aa600074596b9381" candidate_uuid: "30ba8ac122354ff684104423f5b9dfdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d56da4c857e4e8ea564172c26a26744"
I20260808 17:00:58.061960 2599 raft_consensus.cc:3037] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:00:58.061982 2798 raft_consensus.cc:3037] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:00:58.062678 2798 raft_consensus.cc:2454] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30ba8ac122354ff684104423f5b9dfdd in term 1.
I20260808 17:00:58.062733 2599 raft_consensus.cc:2454] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30ba8ac122354ff684104423f5b9dfdd in term 1.
I20260808 17:00:58.062947 2318 leader_election.cc:302] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30ba8ac122354ff684104423f5b9dfdd, f07469e81b09421b85efe529957b07d0; no voters:
I20260808 17:00:58.063086 2888 raft_consensus.cc:2781] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:00:58.063248 2888 raft_consensus.cc:686] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [term 1 LEADER]: Becoming Leader. State: Replica: 30ba8ac122354ff684104423f5b9dfdd, State: Running, Role: LEADER
I20260808 17:00:58.063357 2888 consensus_queue.cc:237] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07469e81b09421b85efe529957b07d0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40689 } } peers { permanent_uuid: "4d56da4c857e4e8ea564172c26a26744" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 32879 } } peers { permanent_uuid: "30ba8ac122354ff684104423f5b9dfdd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43853 } }
I20260808 17:00:58.064208 2211 catalog_manager.cc:5495] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd reported cstate change: term changed from 0 to 1, leader changed from <none> to 30ba8ac122354ff684104423f5b9dfdd (127.8.14.65). New cstate: current_term: 1 leader_uuid: "30ba8ac122354ff684104423f5b9dfdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f07469e81b09421b85efe529957b07d0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d56da4c857e4e8ea564172c26a26744" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30ba8ac122354ff684104423f5b9dfdd" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43853 } health_report { overall_health: HEALTHY } } }
I20260808 17:00:58.073335 2879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:58.071353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58370 (local address 127.8.14.65:43853)
0808 17:00:58.071393 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:58.071395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:58.071404 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:00:58.071468 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:58.071469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:58.071484 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:58.071532 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:00:58.071624 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:58.072035 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:58.072724 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:58.072900 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:58.072950 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:58.073079 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:00:58.073165 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:00:58.073223 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:58.073253 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 17:00:58.073269 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:00:58.074990 2798 raft_consensus.cc:1261] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0 [term 1 FOLLOWER]: Refusing update from remote peer 30ba8ac122354ff684104423f5b9dfdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:00:58.074996 2599 raft_consensus.cc:1261] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744 [term 1 FOLLOWER]: Refusing update from remote peer 30ba8ac122354ff684104423f5b9dfdd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:00:58.075317 2893 consensus_queue.cc:1035] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d56da4c857e4e8ea564172c26a26744" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:00:58.075457 2888 consensus_queue.cc:1035] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f07469e81b09421b85efe529957b07d0" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:00:58.077806 2896 mvcc.cc:205] Tried to move back new op lower bound from 7316309844272656384 to 7316309844228182016. Current Snapshot: MvccSnapshot[applied={T|T < 7316309844272656384}]
I20260808 17:00:58.078152 2897 mvcc.cc:205] Tried to move back new op lower bound from 7316309844272656384 to 7316309844228182016. Current Snapshot: MvccSnapshot[applied={T|T < 7316309844272656384}]
I20260808 17:00:58.078269 2898 mvcc.cc:205] Tried to move back new op lower bound from 7316309844272656384 to 7316309844228182016. Current Snapshot: MvccSnapshot[applied={T|T < 7316309844272656384}]
I20260808 17:00:58.080369 2210 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:00:58.080459 2210 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:00:58.081218 2210 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63cd869c68c34296affd2f48a5f26859: Sending DeleteTablet for 3 replicas of tablet 3f62cf083ad54103aa600074596b9381
I20260808 17:00:58.081566 2766 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f62cf083ad54103aa600074596b9381 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:00:58 UTC) from {username='slave'} at 127.0.0.1:59098
I20260808 17:00:58.081576 2572 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f62cf083ad54103aa600074596b9381 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:00:58 UTC) from {username='slave'} at 127.0.0.1:51606
I20260808 17:00:58.081713 2389 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f62cf083ad54103aa600074596b9381 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:00:58 UTC) from {username='slave'} at 127.0.0.1:58358
I20260808 17:00:58.081919 2914 tablet_replica.cc:330] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd: stopping tablet replica
I20260808 17:00:58.082007 2913 tablet_replica.cc:330] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744: stopping tablet replica
I20260808 17:00:58.082043 2914 raft_consensus.cc:2229] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:00:58.082089 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 2291
I20260808 17:00:58.082118 2913 raft_consensus.cc:2229] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:00:58.082202 2914 raft_consensus.cc:2258] T 3f62cf083ad54103aa600074596b9381 P 30ba8ac122354ff684104423f5b9dfdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:00:58.091295 2913 raft_consensus.cc:2258] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:00:58.092541 2913 ts_tablet_manager.cc:1905] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:00:58.092650 2911 tablet_replica.cc:330] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0: stopping tablet replica
I20260808 17:00:58.092763 2911 raft_consensus.cc:2229] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:00:58.092878 2911 raft_consensus.cc:2258] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:00:58.093307 2911 ts_tablet_manager.cc:1905] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:00:58.094048 2913 ts_tablet_manager.cc:1918] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:00:58.094142 2913 log.cc:1196] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/3f62cf083ad54103aa600074596b9381
I20260808 17:00:58.094410 2913 ts_tablet_manager.cc:1939] T 3f62cf083ad54103aa600074596b9381 P 4d56da4c857e4e8ea564172c26a26744: Deleting consensus metadata
I20260808 17:00:58.094981 2192 catalog_manager.cc:4841] TS 4d56da4c857e4e8ea564172c26a26744 (127.8.14.66:32879): tablet 3f62cf083ad54103aa600074596b9381 (table test-table [id=2754269b18be4135aa855eccb0bd22dd]) successfully deleted
I20260808 17:00:58.095388 2911 ts_tablet_manager.cc:1918] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:00:58.095474 2911 log.cc:1196] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/3f62cf083ad54103aa600074596b9381
I20260808 17:00:58.095885 2911 ts_tablet_manager.cc:1939] T 3f62cf083ad54103aa600074596b9381 P f07469e81b09421b85efe529957b07d0: Deleting consensus metadata
I20260808 17:00:58.096343 2195 catalog_manager.cc:4841] TS f07469e81b09421b85efe529957b07d0 (127.8.14.67:40689): tablet 3f62cf083ad54103aa600074596b9381 (table test-table [id=2754269b18be4135aa855eccb0bd22dd]) successfully deleted
W20260808 17:00:58.098078 2195 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.14.65:43853 (error 108)
I20260808 17:00:58.098387 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 2486
I20260808 17:00:58.098591 2878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:58.098423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:43853 (local address 127.0.0.1:58384)
0808 17:00:58.098503 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:58.098550 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.65:43853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 17:00:58.100096 2195 catalog_manager.cc:4584] TS 30ba8ac122354ff684104423f5b9dfdd (127.8.14.65:43853): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f62cf083ad54103aa600074596b9381: Network error: Client connection negotiation failed: client connection to 127.8.14.65:43853: connect: Connection refused (error 111)
I20260808 17:00:58.107107 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 2673
I20260808 17:00:58.118578 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 2171
2026-08-08T17:00:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (796 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:00:58 dist-test-slave-1nl6 krb5kdc[2920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:00:58 dist-test-slave-1nl6 krb5kdc[2920](info): set up 2 sockets
Aug 08 17:00:58 dist-test-slave-1nl6 krb5kdc[2920](info): commencing operation
krb5kdc: starting...
W20260808 17:00:59.341965 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:00:59 dist-test-slave-1nl6 krb5kdc[2920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:00:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:00:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 17:00:59.459815 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:45101
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:43453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:45101
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:00:59.547071 2938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:59.547267 2938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:59.547290 2938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:59.548461 2938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:00:59.548504 2938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:59.548519 2938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:59.548532 2938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:00:59.548583 2938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:00:59.550035 2938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:43453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:45101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:45101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:59.550280 2938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:59.550545 2938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:59.555480 2943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:59.555459 2949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:59.555518 2947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:59.555685 2938 server_base.cc:1034] running on GCE node
I20260808 17:00:59.555990 2938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:59.556337 2938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:59.557507 2938 hybrid_clock.cc:648] HybridClock initialized: now 1786208459557486 us; error 32 us; skew 500 ppm
Aug 08 17:00:59 dist-test-slave-1nl6 krb5kdc[2920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208459, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:00:59.560055 2938 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 17:00:59.560537 2938 webserver.cc:460] Webserver started at http://127.8.14.126:37621/ using document root <none> and password file <none>
I20260808 17:00:59.560761 2938 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:59.560832 2938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:59.560984 2938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:00:59.562011 2938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "5eed22372e894e87bad19607d06b0f01"
format_stamp: "Formatted at 2026-08-08 17:00:59 on dist-test-slave-1nl6"
server_key: "1de169b65a4de50f04b52388be1d9646"
server_key_iv: "bb1ee57a8075adfedcc2b6f56364325e"
server_key_version: "encryptionkey@0"
I20260808 17:00:59.562388 2938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "5eed22372e894e87bad19607d06b0f01"
format_stamp: "Formatted at 2026-08-08 17:00:59 on dist-test-slave-1nl6"
server_key: "1de169b65a4de50f04b52388be1d9646"
server_key_iv: "bb1ee57a8075adfedcc2b6f56364325e"
server_key_version: "encryptionkey@0"
I20260808 17:00:59.563815 2938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:00:59.564561 2958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:59.564734 2938 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:00:59.564836 2938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "5eed22372e894e87bad19607d06b0f01"
format_stamp: "Formatted at 2026-08-08 17:00:59 on dist-test-slave-1nl6"
server_key: "1de169b65a4de50f04b52388be1d9646"
server_key_iv: "bb1ee57a8075adfedcc2b6f56364325e"
server_key_version: "encryptionkey@0"
I20260808 17:00:59.564919 2938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:59.582263 2938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:59.584062 2938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:59.584219 2938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:59.588567 2938 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:45101
I20260808 17:00:59.588615 3030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:45101 every 8 connection(s)
I20260808 17:00:59.588968 2938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:00:59.589800 3032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:00:59.592127 3032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01: Bootstrap starting.
I20260808 17:00:59.592653 3032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01: Neither blocks nor log segments found. Creating new log.
I20260808 17:00:59.592896 3032 log.cc:824] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01: Log is configured to *not* fsync() on all Append() calls
I20260808 17:00:59.593532 3032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01: No bootstrap required, opened a new log
I20260808 17:00:59.594947 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 2938
I20260808 17:00:59.595073 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:00:59.595261 8249 external_mini_cluster.cc:1428] Setting key 37cb439c7067cf252e9f09a29437bc6c
I20260808 17:00:59.595194 3032 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eed22372e894e87bad19607d06b0f01" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45101 } }
I20260808 17:00:59.595368 3032 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:00:59.595414 3032 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eed22372e894e87bad19607d06b0f01, State: Initialized, Role: FOLLOWER
I20260808 17:00:59.595566 3032 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eed22372e894e87bad19607d06b0f01" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45101 } }
I20260808 17:00:59.595664 3032 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:00:59.595711 3032 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:00:59.595762 3032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:00:59.596964 3032 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eed22372e894e87bad19607d06b0f01" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45101 } }
I20260808 17:00:59.597090 3032 leader_election.cc:302] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5eed22372e894e87bad19607d06b0f01; no voters:
I20260808 17:00:59.597256 3032 leader_election.cc:288] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:00:59.597306 3043 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:00:59.597467 3043 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [term 1 LEADER]: Becoming Leader. State: Replica: 5eed22372e894e87bad19607d06b0f01, State: Running, Role: LEADER
I20260808 17:00:59.597538 3032 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:00:59.597568 3043 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eed22372e894e87bad19607d06b0f01" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45101 } }
I20260808 17:00:59.597942 3043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5eed22372e894e87bad19607d06b0f01. Latest consensus state: current_term: 1 leader_uuid: "5eed22372e894e87bad19607d06b0f01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eed22372e894e87bad19607d06b0f01" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45101 } } }
I20260808 17:00:59.598022 3043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [sys.catalog]: This master's current role is: LEADER
I20260808 17:00:59.598165 3045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5eed22372e894e87bad19607d06b0f01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5eed22372e894e87bad19607d06b0f01" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 45101 } } }
I20260808 17:00:59.598240 3045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01 [sys.catalog]: This master's current role is: LEADER
I20260808 17:00:59.598348 3053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:00:59.598838 3053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 17:00:59 dist-test-slave-1nl6 krb5kdc[2920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:00:59.604238 3053 catalog_manager.cc:1347] Generated new cluster ID: 825ff7c141a74829b683d79af7dc1fe5
I20260808 17:00:59.604303 3053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:00:59.606187 3041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:59.595684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47864 (local address 127.8.14.126:45101)
0808 17:00:59.595895 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:59.595903 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:59.595929 (+ 26us) server_negotiation.cc:407] Connection header received
0808 17:00:59.596055 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:59.596060 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:59.596106 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:59.596215 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:59.597114 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:59.598589 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:59.599699 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:59.600271 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:59.600377 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:59.602627 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:59.602631 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:59.602632 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:00:59.602644 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:59.604722 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:59.605087 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:59.605095 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:59.605095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:59.605187 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:59.605414 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:59.605415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:59.605415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:59.605656 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:59.605824 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:59.605899 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 17:00:59.606038 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":94,"threads_started":1}
I20260808 17:00:59.613914 3053 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:00:59.614583 3053 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:00:59.636283 3053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01: Generated new TSK 0
I20260808 17:00:59.636519 3053 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 17:00:59.673779 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:45101
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:43453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:00:59.771234 3078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:59.771425 3078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:59.771446 3078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:59.772542 3078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:59.772580 3078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:59.772606 3078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 17:00:59.773975 3078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:43453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:45101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:59.774153 3078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:59.774327 3078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:59.779392 3091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:59.779394 3078 server_base.cc:1034] running on GCE node
W20260808 17:00:59.779428 3087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:59.779414 3086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:59.779868 3078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:59.780081 3078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:59.781219 3078 hybrid_clock.cc:648] HybridClock initialized: now 1786208459781205 us; error 31 us; skew 500 ppm
Aug 08 17:00:59 dist-test-slave-1nl6 krb5kdc[2920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208459, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:00:59.788079 3078 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 17:00:59.788515 3078 webserver.cc:460] Webserver started at http://127.8.14.65:37569/ using document root <none> and password file <none>
I20260808 17:00:59.788739 3078 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:59.788810 3078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:59.788961 3078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:00:59.789968 3078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "eddf4e416f7c4b9cbda6ae374459cb10"
format_stamp: "Formatted at 2026-08-08 17:00:59 on dist-test-slave-1nl6"
server_key: "4300df4835ded1cc92b9fdc930d220ad"
server_key_iv: "4be0dcaeb474315510dafa324ebc5b56"
server_key_version: "encryptionkey@0"
I20260808 17:00:59.790339 3078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "eddf4e416f7c4b9cbda6ae374459cb10"
format_stamp: "Formatted at 2026-08-08 17:00:59 on dist-test-slave-1nl6"
server_key: "4300df4835ded1cc92b9fdc930d220ad"
server_key_iv: "4be0dcaeb474315510dafa324ebc5b56"
server_key_version: "encryptionkey@0"
I20260808 17:00:59.791726 3078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:00:59.792469 3103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:59.792641 3078 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:00:59.792748 3078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "eddf4e416f7c4b9cbda6ae374459cb10"
format_stamp: "Formatted at 2026-08-08 17:00:59 on dist-test-slave-1nl6"
server_key: "4300df4835ded1cc92b9fdc930d220ad"
server_key_iv: "4be0dcaeb474315510dafa324ebc5b56"
server_key_version: "encryptionkey@0"
I20260808 17:00:59.792821 3078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:59.805573 3078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:59.807385 3078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:59.807552 3078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:59.807782 3078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:00:59.808087 3078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:00:59.808145 3078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:59.808184 3078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:00:59.808212 3078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:59.814079 3078 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:44091
I20260808 17:00:59.814116 3257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:44091 every 8 connection(s)
I20260808 17:00:59.814427 3078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 17:00:59.819500 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 3078
I20260808 17:00:59.819618 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 17:00:59.819854 8249 external_mini_cluster.cc:1428] Setting key 692af5621ff4fbe6b893d7e31af80a87
Aug 08 17:00:59 dist-test-slave-1nl6 krb5kdc[2920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208459, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:00:59.824316 3041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:59.815111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:55677 (local address 127.8.14.126:45101)
0808 17:00:59.815196 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 17:00:59.815199 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:00:59.815397 (+ 198us) server_negotiation.cc:407] Connection header received
0808 17:00:59.815886 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:00:59.815887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:00:59.815902 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:00:59.815950 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:00:59.816811 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:59.817306 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:59.818245 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:00:59.818494 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:00:59.818550 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:59.821015 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:00:59.821017 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:00:59.821017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:00:59.821025 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:00:59.822763 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:59.823320 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:59.823321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:59.823321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:59.823377 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:00:59.823682 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:00:59.823683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:00:59.823683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:00:59.823788 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:00:59.823847 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 17:00:59.824188 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 17:00:59.824254 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 17:00:59.824604 3263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:00:59.815065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45101 (local address 127.8.14.65:55677)
0808 17:00:59.815257 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 17:00:59.815292 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:00:59.815788 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:00:59.816007 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:00:59.816010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:00:59.816552 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:00:59.816726 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:59.816730 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:59.817386 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:59.817386 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:59.818172 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:00:59.818173 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:00:59.818571 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:00:59.818572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:00:59.818664 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:00:59.819123 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:00:59.819131 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:00:59.820909 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:00:59.822877 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:59.822885 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:59.822887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:59.823232 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:59.823453 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:00:59.823454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:00:59.823454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:00:59.823595 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:00:59.823867 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:00:59.823873 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:00:59.824046 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 17:00:59.824162 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 17:00:59.824343 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":105,"threads_started":1}
I20260808 17:00:59.825101 3258 heartbeater.cc:347] Connected to a master server at 127.8.14.126:45101
I20260808 17:00:59.825217 3258 heartbeater.cc:464] Registering TS with master...
I20260808 17:00:59.825392 3258 heartbeater.cc:511] Master 127.8.14.126:45101 requested a full tablet report, sending...
I20260808 17:00:59.826033 2983 ts_manager.cc:194] Registered new tserver with Master: eddf4e416f7c4b9cbda6ae374459cb10 (127.8.14.65:44091)
I20260808 17:00:59.827035 2983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:55677
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 17:00:59.845763 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:45101
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:43453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:00:59.927070 3267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:00:59.927299 3267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:00:59.927337 3267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:00:59.928458 3267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:00:59.928526 3267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:00:59.928570 3267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 17:00:59.929978 3267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:43453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:45101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:00:59.930240 3267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:00:59.930471 3267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:00:59.935359 3273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:59.935364 3278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:00:59.935371 3274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:00:59.935712 3267 server_base.cc:1034] running on GCE node
I20260808 17:00:59.935925 3267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:00:59.936164 3267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:00:59.937341 3267 hybrid_clock.cc:648] HybridClock initialized: now 1786208459937328 us; error 24 us; skew 500 ppm
Aug 08 17:00:59 dist-test-slave-1nl6 krb5kdc[2920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208459, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:00:59.940084 3267 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 17:00:59.940660 3267 webserver.cc:460] Webserver started at http://127.8.14.66:40419/ using document root <none> and password file <none>
I20260808 17:00:59.940932 3267 fs_manager.cc:359] Metadata directory not provided
I20260808 17:00:59.941018 3267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:00:59.941165 3267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:00:59.942715 3267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "c91e0949636d44939cbc0b482020224d"
format_stamp: "Formatted at 2026-08-08 17:00:59 on dist-test-slave-1nl6"
server_key: "9e6031aa177c5a4b8aedc100167c9a48"
server_key_iv: "24e7b91affb1a441b5ba4a7e79977296"
server_key_version: "encryptionkey@0"
I20260808 17:00:59.943262 3267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "c91e0949636d44939cbc0b482020224d"
format_stamp: "Formatted at 2026-08-08 17:00:59 on dist-test-slave-1nl6"
server_key: "9e6031aa177c5a4b8aedc100167c9a48"
server_key_iv: "24e7b91affb1a441b5ba4a7e79977296"
server_key_version: "encryptionkey@0"
I20260808 17:00:59.945235 3267 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260808 17:00:59.946342 3286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:59.946535 3267 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:00:59.946628 3267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "c91e0949636d44939cbc0b482020224d"
format_stamp: "Formatted at 2026-08-08 17:00:59 on dist-test-slave-1nl6"
server_key: "9e6031aa177c5a4b8aedc100167c9a48"
server_key_iv: "24e7b91affb1a441b5ba4a7e79977296"
server_key_version: "encryptionkey@0"
I20260808 17:00:59.946725 3267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:00:59.995689 3267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:00:59.998492 3267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:00:59.998662 3267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:00:59.998953 3267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:00:59.999331 3267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:00:59.999368 3267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:00:59.999393 3267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:00:59.999439 3267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:00.005426 3267 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:42643
I20260808 17:01:00.005472 3446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:42643 every 8 connection(s)
I20260808 17:01:00.005808 3267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
Aug 08 17:01:00 dist-test-slave-1nl6 krb5kdc[2920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208459, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:01:00.015419 3449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.006495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45101 (local address 127.8.14.66:57171)
0808 17:01:00.006666 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:00.006703 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:00.007289 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:00.007549 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:00.007553 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:00.008147 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:00.008344 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.008349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.009060 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.009061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.009865 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.009867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.010247 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.010248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.010379 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.010830 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:00.010839 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:00.012324 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:00.013727 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:00.013735 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:00.013737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:00.014031 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:00.014284 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:00.014286 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:00.014304 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:00.014474 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:00.014774 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:00.014782 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:00.015023 (+ 241us) client_negotiation.cc:769] Sending connection context
0808 17:01:00.015112 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 17:01:00.015259 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260808 17:01:00.015875 3041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.006488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:57171 (local address 127.8.14.126:45101)
0808 17:01:00.006551 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:00.006554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:00.006794 (+ 240us) server_negotiation.cc:407] Connection header received
0808 17:01:00.007411 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:00.007412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:00.007428 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:00.007480 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:00.008461 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.008961 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.009961 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.010159 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.010240 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.012440 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:00.012442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:00.012442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:00.012450 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:00.013629 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:00.014121 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:00.014122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:00.014122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:00.014213 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:00.014557 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:00.014558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:00.014558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:00.014697 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:00.014763 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:00.015663 (+ 900us) server_negotiation.cc:299] Negotiation successful
0808 17:01:00.015783 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 17:01:00.015887 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 3267
I20260808 17:01:00.016039 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 17:01:00.016251 8249 external_mini_cluster.cc:1428] Setting key b44a1b803d567061a0c7eb2a3c56b062
I20260808 17:01:00.016253 3447 heartbeater.cc:347] Connected to a master server at 127.8.14.126:45101
I20260808 17:01:00.016363 3447 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:00.016561 3447 heartbeater.cc:511] Master 127.8.14.126:45101 requested a full tablet report, sending...
I20260808 17:01:00.016984 2983 ts_manager.cc:194] Registered new tserver with Master: c91e0949636d44939cbc0b482020224d (127.8.14.66:42643)
I20260808 17:01:00.017372 2983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:57171
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 17:01:00.042246 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:45101
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:43453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:00.124068 3453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:00.124261 3453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:00.124284 3453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:00.125406 3453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:00.125449 3453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:00.125502 3453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 17:01:00.127012 3453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:43453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:45101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:00.127250 3453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:00.127501 3453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:00.132473 3462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:00.132481 3467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:00.132550 3464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:00.132668 3453 server_base.cc:1034] running on GCE node
I20260808 17:01:00.132869 3453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:00.133116 3453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:00.134263 3453 hybrid_clock.cc:648] HybridClock initialized: now 1786208460134243 us; error 37 us; skew 500 ppm
Aug 08 17:01:00 dist-test-slave-1nl6 krb5kdc[2920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208460, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:00.142102 3453 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 17:01:00.142652 3453 webserver.cc:460] Webserver started at http://127.8.14.67:44279/ using document root <none> and password file <none>
I20260808 17:01:00.142988 3453 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:00.143081 3453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:00.143234 3453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:00.144721 3453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "f4cad6952e174af8998239f721cb1793"
format_stamp: "Formatted at 2026-08-08 17:01:00 on dist-test-slave-1nl6"
server_key: "a9e623bb2cf107fe33ed90211c418f9d"
server_key_iv: "e28604339b4eb3a73a86760deae343f3"
server_key_version: "encryptionkey@0"
I20260808 17:01:00.145196 3453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "f4cad6952e174af8998239f721cb1793"
format_stamp: "Formatted at 2026-08-08 17:01:00 on dist-test-slave-1nl6"
server_key: "a9e623bb2cf107fe33ed90211c418f9d"
server_key_iv: "e28604339b4eb3a73a86760deae343f3"
server_key_version: "encryptionkey@0"
I20260808 17:01:00.146847 3453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 17:01:00.147671 3475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:00.147864 3453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:00.147960 3453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "f4cad6952e174af8998239f721cb1793"
format_stamp: "Formatted at 2026-08-08 17:01:00 on dist-test-slave-1nl6"
server_key: "a9e623bb2cf107fe33ed90211c418f9d"
server_key_iv: "e28604339b4eb3a73a86760deae343f3"
server_key_version: "encryptionkey@0"
I20260808 17:01:00.148031 3453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:00.168510 3453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:00.170308 3453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:00.170477 3453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:00.170706 3453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:00.171073 3453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:00.171110 3453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:00.171132 3453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:00.171176 3453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:00.177071 3453 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:38693
I20260808 17:01:00.177119 3633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:38693 every 8 connection(s)
I20260808 17:01:00.177423 3453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:01:00 dist-test-slave-1nl6 krb5kdc[2920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208460, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:01:00.185997 3041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.178084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:32895 (local address 127.8.14.126:45101)
0808 17:01:00.178155 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:00.178158 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:00.178429 (+ 271us) server_negotiation.cc:407] Connection header received
0808 17:01:00.179006 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:00.179007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:00.179018 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:00.179070 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:00.179924 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.180388 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.181288 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.181468 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.181521 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.183443 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:00.183445 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:00.183445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:00.183453 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:00.184621 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:00.185022 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:00.185023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:00.185025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:00.185082 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:00.185338 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:00.185339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:00.185339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:00.185454 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:00.185532 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:00.185810 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 17:01:00.185892 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 17:01:00.186190 3637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.178066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:45101 (local address 127.8.14.67:32895)
0808 17:01:00.178270 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:00.178305 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:00.178853 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:00.179134 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:00.179137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:00.179657 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:00.179845 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.179850 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.180464 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.180465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.181208 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.181209 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.181541 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.181542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.181641 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.182039 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:00.182046 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:00.183340 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:00.184707 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:00.184714 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:00.184716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:00.184934 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:00.185158 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:00.185159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:00.185159 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:00.185270 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:00.185524 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:00.185531 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:00.185664 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 17:01:00.185773 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 17:01:00.185922 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":123,"threads_started":1}
I20260808 17:01:00.186637 3634 heartbeater.cc:347] Connected to a master server at 127.8.14.126:45101
I20260808 17:01:00.186757 3634 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:00.187001 3634 heartbeater.cc:511] Master 127.8.14.126:45101 requested a full tablet report, sending...
I20260808 17:01:00.187340 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 3453
I20260808 17:01:00.187444 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 17:01:00.187417 2983 ts_manager.cc:194] Registered new tserver with Master: f4cad6952e174af8998239f721cb1793 (127.8.14.67:38693)
I20260808 17:01:00.187623 8249 external_mini_cluster.cc:1428] Setting key 83cc099106db2dd419c7ba0b366ba5b7
I20260808 17:01:00.187927 2983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:32895
I20260808 17:01:00.188582 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:01:00.195941 3041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.189855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47868 (local address 127.8.14.126:45101)
0808 17:01:00.189915 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:00.189918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:00.190082 (+ 164us) server_negotiation.cc:407] Connection header received
0808 17:01:00.190104 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:00.190104 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:00.190112 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:00.190160 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:00.190647 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.191174 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.191851 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.192101 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.192173 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.193748 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:00.193750 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:00.193750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:00.193757 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:00.194866 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:00.195296 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:00.195297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:00.195297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:00.195352 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:00.195552 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:00.195553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:00.195553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:00.195663 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:00.195742 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:00.195839 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 17:01:00.195887 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 17:01:00.197611 2982 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:01:00.199291 2982 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:01:00.205588 3655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.201497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:44091 (local address 127.0.0.1:47068)
0808 17:01:00.201709 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:00.201717 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:00.201756 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:00.202109 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:00.202112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:00.202114 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:00.202186 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.202188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.203560 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.203560 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.204586 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.204589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.205390 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.205391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.205445 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.205455 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 17:01:00.205505 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 17:01:00.205539 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":102,"thread_start_us":95,"threads_started":1}
I20260808 17:01:00.205664 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.201733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47068 (local address 127.8.14.65:44091)
0808 17:01:00.201935 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:00.201939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:00.201951 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:01:00.201990 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:00.201992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:00.202014 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:00.202081 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:00.202272 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.203424 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.204676 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.205308 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.205385 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.205440 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:00.205531 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 17:01:00.205569 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":78,"threads_started":1}
I20260808 17:01:00.206346 3168 tablet_service.cc:1467] Processing CreateTablet for tablet 6609bfee7cdc4393a17722a82afe9b8c (DEFAULT_TABLE table=test-table [id=4b6f726dc6ad443c920e582632a78da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:00.206736 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.201469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:42643 (local address 127.0.0.1:50364)
0808 17:01:00.201656 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:00.201672 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:00.201848 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:00.202183 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:00.202184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:00.202187 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:00.202235 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.202238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.203656 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.203656 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.204842 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.204845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.206416 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.206417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.206482 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.206486 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:01:00.206601 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 17:01:00.206645 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":35,"thread_start_us":94,"threads_started":1}
I20260808 17:01:00.206801 3168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6609bfee7cdc4393a17722a82afe9b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:00.207157 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.201638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37106 (local address 127.8.14.67:38693)
0808 17:01:00.202219 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:00.202223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:00.202231 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:01:00.202262 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:00.202264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:00.202279 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:00.202518 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:00.202601 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.203387 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.204937 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.205609 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.206495 (+ 886us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.206553 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:00.207068 (+ 515us) server_negotiation.cc:299] Negotiation successful
0808 17:01:00.207100 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":71,"threads_started":1}
I20260808 17:01:00.207175 3659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.201800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50364 (local address 127.8.14.66:42643)
0808 17:01:00.201997 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:00.202001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:00.202009 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:01:00.202044 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:00.202045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:00.202076 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:00.202162 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:00.202310 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.203495 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.205695 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.206325 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.206387 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.206420 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:00.207048 (+ 628us) server_negotiation.cc:299] Negotiation successful
0808 17:01:00.207093 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":100,"threads_started":1}
I20260808 17:01:00.206853 3653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.201469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:38693 (local address 127.0.0.1:37106)
0808 17:01:00.201635 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:00.201672 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:00.201737 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:00.202394 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:00.202395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:00.202397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:00.202437 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.202438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.203491 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.203492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.204859 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.204861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.206507 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.206508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.206574 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.206578 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:01:00.206660 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 17:01:00.206709 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":65,"thread_start_us":72,"threads_started":1}
I20260808 17:01:00.207861 3543 tablet_service.cc:1467] Processing CreateTablet for tablet 6609bfee7cdc4393a17722a82afe9b8c (DEFAULT_TABLE table=test-table [id=4b6f726dc6ad443c920e582632a78da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:00.207929 3353 tablet_service.cc:1467] Processing CreateTablet for tablet 6609bfee7cdc4393a17722a82afe9b8c (DEFAULT_TABLE table=test-table [id=4b6f726dc6ad443c920e582632a78da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:00.208175 3543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6609bfee7cdc4393a17722a82afe9b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:00.208264 3353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6609bfee7cdc4393a17722a82afe9b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:00.210438 3660 tablet_bootstrap.cc:492] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10: Bootstrap starting.
I20260808 17:01:00.210927 3661 tablet_bootstrap.cc:492] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793: Bootstrap starting.
I20260808 17:01:00.211294 3660 tablet_bootstrap.cc:654] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:00.211658 3661 tablet_bootstrap.cc:654] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:00.211663 3665 tablet_bootstrap.cc:492] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d: Bootstrap starting.
I20260808 17:01:00.211998 3660 log.cc:824] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:00.212312 3661 log.cc:824] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:00.212438 3665 tablet_bootstrap.cc:654] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:00.212898 3660 tablet_bootstrap.cc:492] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10: No bootstrap required, opened a new log
I20260808 17:01:00.212980 3660 ts_tablet_manager.cc:1397] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 17:01:00.213028 3665 log.cc:824] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:00.213771 3665 tablet_bootstrap.cc:492] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d: No bootstrap required, opened a new log
I20260808 17:01:00.213781 3661 tablet_bootstrap.cc:492] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793: No bootstrap required, opened a new log
I20260808 17:01:00.213851 3661 ts_tablet_manager.cc:1397] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 17:01:00.213863 3665 ts_tablet_manager.cc:1397] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:01:00.214841 3660 raft_consensus.cc:348] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4cad6952e174af8998239f721cb1793" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 38693 } } peers { permanent_uuid: "c91e0949636d44939cbc0b482020224d" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42643 } } peers { permanent_uuid: "eddf4e416f7c4b9cbda6ae374459cb10" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 44091 } }
I20260808 17:01:00.215060 3660 raft_consensus.cc:374] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:00.215108 3660 raft_consensus.cc:729] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eddf4e416f7c4b9cbda6ae374459cb10, State: Initialized, Role: FOLLOWER
I20260808 17:01:00.215219 3660 consensus_queue.cc:260] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4cad6952e174af8998239f721cb1793" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 38693 } } peers { permanent_uuid: "c91e0949636d44939cbc0b482020224d" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42643 } } peers { permanent_uuid: "eddf4e416f7c4b9cbda6ae374459cb10" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 44091 } }
I20260808 17:01:00.215521 3660 ts_tablet_manager.cc:1428] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:01:00.215531 3258 heartbeater.cc:503] Master 127.8.14.126:45101 was elected leader, sending a full tablet report...
I20260808 17:01:00.215768 3665 raft_consensus.cc:348] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4cad6952e174af8998239f721cb1793" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 38693 } } peers { permanent_uuid: "c91e0949636d44939cbc0b482020224d" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42643 } } peers { permanent_uuid: "eddf4e416f7c4b9cbda6ae374459cb10" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 44091 } }
I20260808 17:01:00.215849 3661 raft_consensus.cc:348] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4cad6952e174af8998239f721cb1793" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 38693 } } peers { permanent_uuid: "c91e0949636d44939cbc0b482020224d" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42643 } } peers { permanent_uuid: "eddf4e416f7c4b9cbda6ae374459cb10" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 44091 } }
I20260808 17:01:00.215973 3665 raft_consensus.cc:374] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:00.215986 3661 raft_consensus.cc:374] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:00.216023 3661 raft_consensus.cc:729] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4cad6952e174af8998239f721cb1793, State: Initialized, Role: FOLLOWER
I20260808 17:01:00.216023 3665 raft_consensus.cc:729] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c91e0949636d44939cbc0b482020224d, State: Initialized, Role: FOLLOWER
I20260808 17:01:00.216126 3665 consensus_queue.cc:260] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4cad6952e174af8998239f721cb1793" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 38693 } } peers { permanent_uuid: "c91e0949636d44939cbc0b482020224d" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42643 } } peers { permanent_uuid: "eddf4e416f7c4b9cbda6ae374459cb10" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 44091 } }
I20260808 17:01:00.216140 3661 consensus_queue.cc:260] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4cad6952e174af8998239f721cb1793" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 38693 } } peers { permanent_uuid: "c91e0949636d44939cbc0b482020224d" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42643 } } peers { permanent_uuid: "eddf4e416f7c4b9cbda6ae374459cb10" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 44091 } }
I20260808 17:01:00.216403 3661 ts_tablet_manager.cc:1428] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:01:00.216400 3665 ts_tablet_manager.cc:1428] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:01:00.216516 3447 heartbeater.cc:503] Master 127.8.14.126:45101 was elected leader, sending a full tablet report...
I20260808 17:01:00.216547 3634 heartbeater.cc:503] Master 127.8.14.126:45101 was elected leader, sending a full tablet report...
W20260808 17:01:00.256363 3448 tablet.cc:2369] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:01:00.295118 3670 raft_consensus.cc:482] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:01:00.295246 3670 raft_consensus.cc:504] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4cad6952e174af8998239f721cb1793" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 38693 } } peers { permanent_uuid: "c91e0949636d44939cbc0b482020224d" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42643 } } peers { permanent_uuid: "eddf4e416f7c4b9cbda6ae374459cb10" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 44091 } }
I20260808 17:01:00.295563 3670 leader_election.cc:288] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4cad6952e174af8998239f721cb1793 (127.8.14.67:38693), c91e0949636d44939cbc0b482020224d (127.8.14.66:42643)
I20260808 17:01:00.299589 3659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.295893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:45133 (local address 127.8.14.66:42643)
0808 17:01:00.295959 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:00.295961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:00.295969 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:01:00.295997 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:00.295998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:00.296011 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:00.296064 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:00.296286 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.297079 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.298589 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.299238 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.299313 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.299324 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:00.299483 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 17:01:00.299518 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 17:01:00.299589 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.296004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:57785 (local address 127.8.14.67:38693)
0808 17:01:00.296080 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:00.296081 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:00.296102 (+ 21us) server_negotiation.cc:407] Connection header received
0808 17:01:00.296126 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:00.296127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:00.296142 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:00.296217 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:00.296407 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.297080 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.298589 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.299238 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.299313 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.299323 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:00.299482 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 17:01:00.299519 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 17:01:00.299626 3263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.295736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:42643 (local address 127.8.14.65:45133)
0808 17:01:00.295796 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:00.295810 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:00.295878 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:00.296078 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:00.296080 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:00.296081 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:00.296205 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.296207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.297162 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.297163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.298493 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.298497 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.299319 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.299320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.299399 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.299406 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 17:01:00.299462 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 17:01:00.299512 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":45}
I20260808 17:01:00.299633 3674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.295736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:38693 (local address 127.8.14.65:57785)
0808 17:01:00.295906 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:00.295917 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:00.295968 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:00.296252 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:00.296253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:00.296255 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:00.296336 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.296338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.297162 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.297163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.298508 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.298510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.299325 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.299326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.299399 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.299407 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 17:01:00.299462 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 17:01:00.299512 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":92,"thread_start_us":96,"threads_started":1}
I20260808 17:01:00.300010 3379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6609bfee7cdc4393a17722a82afe9b8c" candidate_uuid: "eddf4e416f7c4b9cbda6ae374459cb10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c91e0949636d44939cbc0b482020224d" is_pre_election: true
I20260808 17:01:00.300010 3573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6609bfee7cdc4393a17722a82afe9b8c" candidate_uuid: "eddf4e416f7c4b9cbda6ae374459cb10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4cad6952e174af8998239f721cb1793" is_pre_election: true
I20260808 17:01:00.300153 3573 raft_consensus.cc:2454] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eddf4e416f7c4b9cbda6ae374459cb10 in term 0.
I20260808 17:01:00.300172 3379 raft_consensus.cc:2454] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eddf4e416f7c4b9cbda6ae374459cb10 in term 0.
I20260808 17:01:00.300375 3108 leader_election.cc:302] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eddf4e416f7c4b9cbda6ae374459cb10, f4cad6952e174af8998239f721cb1793; no voters:
I20260808 17:01:00.300529 3670 raft_consensus.cc:2781] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:01:00.300587 3670 raft_consensus.cc:482] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:01:00.300611 3670 raft_consensus.cc:3037] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:00.301376 3670 raft_consensus.cc:504] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4cad6952e174af8998239f721cb1793" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 38693 } } peers { permanent_uuid: "c91e0949636d44939cbc0b482020224d" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42643 } } peers { permanent_uuid: "eddf4e416f7c4b9cbda6ae374459cb10" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 44091 } }
I20260808 17:01:00.301528 3670 leader_election.cc:288] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [CANDIDATE]: Term 1 election: Requested vote from peers f4cad6952e174af8998239f721cb1793 (127.8.14.67:38693), c91e0949636d44939cbc0b482020224d (127.8.14.66:42643)
I20260808 17:01:00.301703 3379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6609bfee7cdc4393a17722a82afe9b8c" candidate_uuid: "eddf4e416f7c4b9cbda6ae374459cb10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c91e0949636d44939cbc0b482020224d"
I20260808 17:01:00.301707 3573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6609bfee7cdc4393a17722a82afe9b8c" candidate_uuid: "eddf4e416f7c4b9cbda6ae374459cb10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4cad6952e174af8998239f721cb1793"
I20260808 17:01:00.301786 3379 raft_consensus.cc:3037] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:00.301786 3573 raft_consensus.cc:3037] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:00.302686 3573 raft_consensus.cc:2454] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eddf4e416f7c4b9cbda6ae374459cb10 in term 1.
I20260808 17:01:00.302686 3379 raft_consensus.cc:2454] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eddf4e416f7c4b9cbda6ae374459cb10 in term 1.
I20260808 17:01:00.302860 3108 leader_election.cc:302] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eddf4e416f7c4b9cbda6ae374459cb10, f4cad6952e174af8998239f721cb1793; no voters:
I20260808 17:01:00.302968 3670 raft_consensus.cc:2781] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:00.303138 3670 raft_consensus.cc:686] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [term 1 LEADER]: Becoming Leader. State: Replica: eddf4e416f7c4b9cbda6ae374459cb10, State: Running, Role: LEADER
I20260808 17:01:00.303237 3670 consensus_queue.cc:237] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4cad6952e174af8998239f721cb1793" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 38693 } } peers { permanent_uuid: "c91e0949636d44939cbc0b482020224d" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42643 } } peers { permanent_uuid: "eddf4e416f7c4b9cbda6ae374459cb10" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 44091 } }
I20260808 17:01:00.304148 2983 catalog_manager.cc:5495] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 reported cstate change: term changed from 0 to 1, leader changed from <none> to eddf4e416f7c4b9cbda6ae374459cb10 (127.8.14.65). New cstate: current_term: 1 leader_uuid: "eddf4e416f7c4b9cbda6ae374459cb10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4cad6952e174af8998239f721cb1793" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 38693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c91e0949636d44939cbc0b482020224d" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eddf4e416f7c4b9cbda6ae374459cb10" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 44091 } health_report { overall_health: HEALTHY } } }
W20260808 17:01:00.315160 3261 tablet.cc:2369] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:01:00.321714 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.319235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47076 (local address 127.8.14.65:44091)
0808 17:01:00.319284 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:00.319286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:00.319295 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:01:00.319360 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:00.319361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:00.319380 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:00.319483 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:00.319624 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.320107 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.320881 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.321092 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.321169 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.321319 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:00.321434 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:00.321559 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:00.321611 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 17:01:00.321620 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 17:01:00.323493 3379 raft_consensus.cc:1261] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d [term 1 FOLLOWER]: Refusing update from remote peer eddf4e416f7c4b9cbda6ae374459cb10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:00.323518 3573 raft_consensus.cc:1261] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793 [term 1 FOLLOWER]: Refusing update from remote peer eddf4e416f7c4b9cbda6ae374459cb10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:00.323880 3670 consensus_queue.cc:1035] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c91e0949636d44939cbc0b482020224d" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 42643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:00.324024 3677 consensus_queue.cc:1035] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4cad6952e174af8998239f721cb1793" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 38693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:00.326334 3685 mvcc.cc:205] Tried to move back new op lower bound from 7316309853482033152 to 7316309853402959872. Current Snapshot: MvccSnapshot[applied={T|T < 7316309853482033152}]
I20260808 17:01:00.327098 3688 mvcc.cc:205] Tried to move back new op lower bound from 7316309853482033152 to 7316309853402959872. Current Snapshot: MvccSnapshot[applied={T|T < 7316309853482033152}]
I20260808 17:01:00.328922 2982 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:00.329026 2982 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:00.329782 2982 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5eed22372e894e87bad19607d06b0f01: Sending DeleteTablet for 3 replicas of tablet 6609bfee7cdc4393a17722a82afe9b8c
I20260808 17:01:00.330130 3168 tablet_service.cc:1514] Processing DeleteTablet for tablet 6609bfee7cdc4393a17722a82afe9b8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:00 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:47068
I20260808 17:01:00.330157 3543 tablet_service.cc:1514] Processing DeleteTablet for tablet 6609bfee7cdc4393a17722a82afe9b8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:00 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:37106
I20260808 17:01:00.330189 3353 tablet_service.cc:1514] Processing DeleteTablet for tablet 6609bfee7cdc4393a17722a82afe9b8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:00 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:50364
I20260808 17:01:00.330376 3699 tablet_replica.cc:330] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793: stopping tablet replica
I20260808 17:01:00.330475 3699 raft_consensus.cc:2229] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:00.330475 3698 tablet_replica.cc:330] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10: stopping tablet replica
I20260808 17:01:00.330566 3699 raft_consensus.cc:2258] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:00.330567 3698 raft_consensus.cc:2229] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:01:00.330714 3698 raft_consensus.cc:2258] T 6609bfee7cdc4393a17722a82afe9b8c P eddf4e416f7c4b9cbda6ae374459cb10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:00.330750 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 3078
I20260808 17:01:00.331067 3699 ts_tablet_manager.cc:1905] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:00.333319 3699 ts_tablet_manager.cc:1918] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:01:00.333392 3699 log.cc:1196] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/6609bfee7cdc4393a17722a82afe9b8c
I20260808 17:01:00.333477 3700 tablet_replica.cc:330] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d: stopping tablet replica
I20260808 17:01:00.333573 3700 raft_consensus.cc:2229] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:00.333665 3699 ts_tablet_manager.cc:1939] T 6609bfee7cdc4393a17722a82afe9b8c P f4cad6952e174af8998239f721cb1793: Deleting consensus metadata
I20260808 17:01:00.333681 3700 raft_consensus.cc:2258] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:00.334040 2965 catalog_manager.cc:4841] TS f4cad6952e174af8998239f721cb1793 (127.8.14.67:38693): tablet 6609bfee7cdc4393a17722a82afe9b8c (table test-table [id=4b6f726dc6ad443c920e582632a78da5]) successfully deleted
I20260808 17:01:00.336493 3700 ts_tablet_manager.cc:1905] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:00.337597 3700 ts_tablet_manager.cc:1918] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:01:00.337653 3700 log.cc:1196] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/6609bfee7cdc4393a17722a82afe9b8c
I20260808 17:01:00.337844 3700 ts_tablet_manager.cc:1939] T 6609bfee7cdc4393a17722a82afe9b8c P c91e0949636d44939cbc0b482020224d: Deleting consensus metadata
I20260808 17:01:00.338198 2962 catalog_manager.cc:4841] TS c91e0949636d44939cbc0b482020224d (127.8.14.66:42643): tablet 6609bfee7cdc4393a17722a82afe9b8c (table test-table [id=4b6f726dc6ad443c920e582632a78da5]) successfully deleted
W20260808 17:01:00.340204 2963 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.14.65:44091 (error 108)
I20260808 17:01:00.340695 3654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.340556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:44091 (local address 127.0.0.1:47084)
0808 17:01:00.340610 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:00.340649 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.65:44091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 17:01:00.340718 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 3267
W20260808 17:01:00.340853 2963 catalog_manager.cc:4584] TS eddf4e416f7c4b9cbda6ae374459cb10 (127.8.14.65:44091): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6609bfee7cdc4393a17722a82afe9b8c: Network error: Client connection negotiation failed: client connection to 127.8.14.65:44091: connect: Connection refused (error 111)
I20260808 17:01:00.348315 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 3453
I20260808 17:01:00.359752 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 2938
2026-08-08T17:01:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2242 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T17:01:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:00Z Disabled control of system clock
I20260808 17:01:00.393604 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:35175
--webserver_interface=127.8.14.126
--webserver_port=0
--builtin_ntp_servers=127.8.14.84:41689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:35175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:01:00.474229 3717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:00.474433 3717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:00.474452 3717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:00.475638 3717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:00.475680 3717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:00.475693 3717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:00.475705 3717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:00.475752 3717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:00.477098 3717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:35175
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:35175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:00.477360 3717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:00.477653 3717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:00.482453 3727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:00.482453 3729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:00.482577 3717 server_base.cc:1034] running on GCE node
W20260808 17:01:00.482659 3725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:00.482933 3717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:00.483268 3717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:00.484421 3717 hybrid_clock.cc:648] HybridClock initialized: now 1786208460484396 us; error 41 us; skew 500 ppm
I20260808 17:01:00.485807 3717 webserver.cc:460] Webserver started at http://127.8.14.126:41707/ using document root <none> and password file <none>
I20260808 17:01:00.486063 3717 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:00.486109 3717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:00.486263 3717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:00.487439 3717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "5edb265373a2467d83780b2850de4371"
format_stamp: "Formatted at 2026-08-08 17:01:00 on dist-test-slave-1nl6"
server_key: "3d02feb8ea772c52a6ea700c384bb817"
server_key_iv: "712d5ccbac6c86e59acf5d0a0ba00eb3"
server_key_version: "encryptionkey@0"
I20260808 17:01:00.487828 3717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "5edb265373a2467d83780b2850de4371"
format_stamp: "Formatted at 2026-08-08 17:01:00 on dist-test-slave-1nl6"
server_key: "3d02feb8ea772c52a6ea700c384bb817"
server_key_iv: "712d5ccbac6c86e59acf5d0a0ba00eb3"
server_key_version: "encryptionkey@0"
I20260808 17:01:00.489115 3717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:01:00.489825 3736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:00.490005 3717 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:00.490080 3717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "5edb265373a2467d83780b2850de4371"
format_stamp: "Formatted at 2026-08-08 17:01:00 on dist-test-slave-1nl6"
server_key: "3d02feb8ea772c52a6ea700c384bb817"
server_key_iv: "712d5ccbac6c86e59acf5d0a0ba00eb3"
server_key_version: "encryptionkey@0"
I20260808 17:01:00.490173 3717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:00.513092 3717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:00.513442 3717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:00.513607 3717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:00.517897 3717 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:35175
I20260808 17:01:00.517956 3807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:35175 every 8 connection(s)
I20260808 17:01:00.518332 3717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:00.518608 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 3717
I20260808 17:01:00.518731 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:00.519037 8249 external_mini_cluster.cc:1428] Setting key 1728d492c05d06788cc05a261261923d
I20260808 17:01:00.519675 3808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:00.522974 3808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371: Bootstrap starting.
I20260808 17:01:00.523670 3808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:00.523977 3808 log.cc:824] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:00.524818 3808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371: No bootstrap required, opened a new log
I20260808 17:01:00.525128 3814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.519579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35980 (local address 127.8.14.126:35175)
0808 17:01:00.519875 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:00.519884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:00.519916 (+ 32us) server_negotiation.cc:407] Connection header received
0808 17:01:00.520461 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:00.520471 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:00.520554 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:00.520708 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:00.520878 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.522218 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.522973 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.523571 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.523703 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.524387 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:00.524391 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:00.524391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:00.524595 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:00.524638 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:00.524653 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:00.524721 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:00.524794 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 17:01:00.524926 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":106,"threads_started":1}
I20260808 17:01:00.526870 3808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edb265373a2467d83780b2850de4371" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35175 } }
I20260808 17:01:00.527079 3808 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:00.527108 3808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edb265373a2467d83780b2850de4371, State: Initialized, Role: FOLLOWER
I20260808 17:01:00.527191 3808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edb265373a2467d83780b2850de4371" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35175 } }
I20260808 17:01:00.527241 3808 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:00.527261 3808 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:00.527293 3808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:00.527776 3808 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edb265373a2467d83780b2850de4371" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35175 } }
I20260808 17:01:00.527868 3808 leader_election.cc:302] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5edb265373a2467d83780b2850de4371; no voters:
I20260808 17:01:00.528002 3808 leader_election.cc:288] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:00.528071 3817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:00.528237 3817 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [term 1 LEADER]: Becoming Leader. State: Replica: 5edb265373a2467d83780b2850de4371, State: Running, Role: LEADER
I20260808 17:01:00.528295 3808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:00.528324 3817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edb265373a2467d83780b2850de4371" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35175 } }
I20260808 17:01:00.528661 3820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5edb265373a2467d83780b2850de4371" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edb265373a2467d83780b2850de4371" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35175 } } }
I20260808 17:01:00.528774 3820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:00.528661 3821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5edb265373a2467d83780b2850de4371. Latest consensus state: current_term: 1 leader_uuid: "5edb265373a2467d83780b2850de4371" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edb265373a2467d83780b2850de4371" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 35175 } } }
I20260808 17:01:00.528838 3821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:00.529165 3826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:00.529686 3826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:00.531328 3826 catalog_manager.cc:1347] Generated new cluster ID: a41777d7853f4da2aaeb34c57a2ec3d5
I20260808 17:01:00.531388 3826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:00.559429 3826 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:00.559959 3826 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:00.575362 3826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371: Generated new TSK 0
I20260808 17:01:00.575554 3826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:00.581096 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35175
--builtin_ntp_servers=127.8.14.84:41689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:01:00.679625 3843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:00.679804 3843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:00.679826 3843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:00.680860 3843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:00.680902 3843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:00.680925 3843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 17:01:00.682236 3843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:00.682436 3843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:00.682659 3843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:00.687767 3853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:00.687767 3854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:00.687983 3843 server_base.cc:1034] running on GCE node
W20260808 17:01:00.688062 3856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:00.688293 3843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:00.688549 3843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:00.689713 3843 hybrid_clock.cc:648] HybridClock initialized: now 1786208460689710 us; error 35 us; skew 500 ppm
I20260808 17:01:00.690876 3843 webserver.cc:460] Webserver started at http://127.8.14.65:41173/ using document root <none> and password file <none>
I20260808 17:01:00.691180 3843 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:00.691248 3843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:00.691421 3843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:00.692482 3843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "1c05eebc5600425098470f97101de51b"
format_stamp: "Formatted at 2026-08-08 17:01:00 on dist-test-slave-1nl6"
server_key: "9cd5c7036d6813aa9136a5b980e099a0"
server_key_iv: "0a1d77fad369bf2bd072d466ea32f320"
server_key_version: "encryptionkey@0"
I20260808 17:01:00.692866 3843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "1c05eebc5600425098470f97101de51b"
format_stamp: "Formatted at 2026-08-08 17:01:00 on dist-test-slave-1nl6"
server_key: "9cd5c7036d6813aa9136a5b980e099a0"
server_key_iv: "0a1d77fad369bf2bd072d466ea32f320"
server_key_version: "encryptionkey@0"
I20260808 17:01:00.694268 3843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:01:00.695060 3864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:00.695228 3843 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:00.695348 3843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "1c05eebc5600425098470f97101de51b"
format_stamp: "Formatted at 2026-08-08 17:01:00 on dist-test-slave-1nl6"
server_key: "9cd5c7036d6813aa9136a5b980e099a0"
server_key_iv: "0a1d77fad369bf2bd072d466ea32f320"
server_key_version: "encryptionkey@0"
I20260808 17:01:00.695456 3843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:00.713581 3843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:00.713891 3843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:00.714043 3843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:00.714284 3843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:00.714614 3843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:00.714670 3843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:00.714711 3843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:00.714740 3843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:00.720616 3843 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:37643
I20260808 17:01:00.720692 4024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:37643 every 8 connection(s)
I20260808 17:01:00.721009 3843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 17:01:00.726248 3814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.721766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:42619 (local address 127.8.14.126:35175)
0808 17:01:00.721824 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:00.721826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:00.722046 (+ 220us) server_negotiation.cc:407] Connection header received
0808 17:01:00.722514 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:00.722514 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:00.722527 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:00.722588 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:00.723293 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.723770 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.724664 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.724848 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.724895 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.725941 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:00.725957 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:00.725957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:00.725963 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:00.725981 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:00.725985 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:00.726073 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:00.726157 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 17:01:00.726191 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 17:01:00.726446 4027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.721714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:35175 (local address 127.8.14.65:42619)
0808 17:01:00.721917 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:00.721956 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:00.722432 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:00.722637 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:00.722640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:00.722648 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:00.723216 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.723222 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.723855 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.723856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.724590 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.724592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.724918 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.724919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.725081 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.725750 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:01:00.725751 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:00.725792 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:01:00.725823 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:00.726078 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:00.726084 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:00.726087 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:00.726155 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 17:01:00.726232 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260808 17:01:00.726979 4025 heartbeater.cc:347] Connected to a master server at 127.8.14.126:35175
I20260808 17:01:00.727089 4025 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:00.727272 4025 heartbeater.cc:511] Master 127.8.14.126:35175 requested a full tablet report, sending...
I20260808 17:01:00.727775 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 3843
I20260808 17:01:00.727893 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 17:01:00.727918 3759 ts_manager.cc:194] Registered new tserver with Master: 1c05eebc5600425098470f97101de51b (127.8.14.65:37643)
I20260808 17:01:00.728120 8249 external_mini_cluster.cc:1428] Setting key b6ffed2947423980bb1c8f93aacab38a
I20260808 17:01:00.728761 3759 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.14.65:42619
I20260808 17:01:00.729398 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35175
--builtin_ntp_servers=127.8.14.84:41689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:01:00.811029 4028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:00.811230 4028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:00.811252 4028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:00.812351 4028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:00.812387 4028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:00.812412 4028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 17:01:00.813660 4028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:00.813833 4028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:00.814055 4028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:00.818881 4037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:00.818892 4035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:00.818924 4039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:00.819154 4028 server_base.cc:1034] running on GCE node
I20260808 17:01:00.819387 4028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:00.819656 4028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:00.820820 4028 hybrid_clock.cc:648] HybridClock initialized: now 1786208460820809 us; error 37 us; skew 500 ppm
I20260808 17:01:00.822094 4028 webserver.cc:460] Webserver started at http://127.8.14.66:42151/ using document root <none> and password file <none>
I20260808 17:01:00.822333 4028 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:00.822403 4028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:00.822538 4028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:00.823587 4028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "eaba7dc432c445ac84e923b217a01988"
format_stamp: "Formatted at 2026-08-08 17:01:00 on dist-test-slave-1nl6"
server_key: "75a98abffd6de42784189187916fc34c"
server_key_iv: "e213a92bce18403b4de466dd3d21f7e0"
server_key_version: "encryptionkey@0"
I20260808 17:01:00.823980 4028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "eaba7dc432c445ac84e923b217a01988"
format_stamp: "Formatted at 2026-08-08 17:01:00 on dist-test-slave-1nl6"
server_key: "75a98abffd6de42784189187916fc34c"
server_key_iv: "e213a92bce18403b4de466dd3d21f7e0"
server_key_version: "encryptionkey@0"
I20260808 17:01:00.825284 4028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:01:00.825958 4049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:00.826139 4028 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:00.826252 4028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "eaba7dc432c445ac84e923b217a01988"
format_stamp: "Formatted at 2026-08-08 17:01:00 on dist-test-slave-1nl6"
server_key: "75a98abffd6de42784189187916fc34c"
server_key_iv: "e213a92bce18403b4de466dd3d21f7e0"
server_key_version: "encryptionkey@0"
I20260808 17:01:00.826330 4028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:00.844728 4028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:00.845026 4028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:00.845177 4028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:00.845511 4028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:00.845870 4028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:00.845928 4028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:00.845958 4028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:00.845988 4028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:00.851730 4028 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:38761
I20260808 17:01:00.851768 4201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:38761 every 8 connection(s)
I20260808 17:01:00.852095 4028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 17:01:00.855116 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 4028
I20260808 17:01:00.855227 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 17:01:00.855415 8249 external_mini_cluster.cc:1428] Setting key 5f83a095d747ce0dae32bbadbb45e966
I20260808 17:01:00.856423 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35175
--builtin_ntp_servers=127.8.14.84:41689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 17:01:00.857784 3814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.852896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:35179 (local address 127.8.14.126:35175)
0808 17:01:00.852973 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:00.852976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:00.853176 (+ 200us) server_negotiation.cc:407] Connection header received
0808 17:01:00.853629 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:00.853630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:00.853641 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:00.853691 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:00.854420 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.854924 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.856329 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.856576 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.856641 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.857487 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:00.857489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:00.857490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:00.857494 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:00.857516 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:00.857521 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:00.857582 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:00.857677 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 17:01:00.857705 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 17:01:00.858038 4204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.852880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:35175 (local address 127.8.14.66:35179)
0808 17:01:00.853064 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:00.853095 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:00.853545 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:00.853747 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:00.853750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:00.853757 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:00.854238 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.854242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.855024 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.855025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.856228 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.856231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.856652 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.856653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.856790 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.857348 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:01:00.857349 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:00.857375 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:01:00.857397 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:00.857583 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:00.857590 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:00.857593 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:00.857677 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 17:01:00.857776 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260808 17:01:00.858515 4202 heartbeater.cc:347] Connected to a master server at 127.8.14.126:35175
I20260808 17:01:00.858608 4202 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:00.858798 4202 heartbeater.cc:511] Master 127.8.14.126:35175 requested a full tablet report, sending...
I20260808 17:01:00.859210 3759 ts_manager.cc:194] Registered new tserver with Master: eaba7dc432c445ac84e923b217a01988 (127.8.14.66:38761)
I20260808 17:01:00.859591 3759 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.14.66:35179
W20260808 17:01:00.943599 4205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:00.943787 4205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:00.943807 4205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:00.944900 4205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:00.944938 4205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:00.944989 4205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 17:01:00.946324 4205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:35175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:00.946574 4205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:00.946870 4205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:00.951720 4212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:00.953001 4216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:00.953018 4205 server_base.cc:1034] running on GCE node
W20260808 17:01:00.953403 4214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:00.953686 4205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:00.953939 4205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:00.955098 4205 hybrid_clock.cc:648] HybridClock initialized: now 1786208460955080 us; error 30 us; skew 500 ppm
I20260808 17:01:00.956278 4205 webserver.cc:460] Webserver started at http://127.8.14.67:42501/ using document root <none> and password file <none>
I20260808 17:01:00.956545 4205 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:00.956617 4205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:00.956776 4205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:00.957801 4205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "691f0c68105944ab9a778857533442e6"
format_stamp: "Formatted at 2026-08-08 17:01:00 on dist-test-slave-1nl6"
server_key: "8dae063c8bd58ec7fce4e8384113cb08"
server_key_iv: "bf6122c98ff09d216b9ed40f9a9d661a"
server_key_version: "encryptionkey@0"
I20260808 17:01:00.958176 4205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "691f0c68105944ab9a778857533442e6"
format_stamp: "Formatted at 2026-08-08 17:01:00 on dist-test-slave-1nl6"
server_key: "8dae063c8bd58ec7fce4e8384113cb08"
server_key_iv: "bf6122c98ff09d216b9ed40f9a9d661a"
server_key_version: "encryptionkey@0"
I20260808 17:01:00.959534 4205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:01:00.960209 4223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:00.960382 4205 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:00.960492 4205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "691f0c68105944ab9a778857533442e6"
format_stamp: "Formatted at 2026-08-08 17:01:00 on dist-test-slave-1nl6"
server_key: "8dae063c8bd58ec7fce4e8384113cb08"
server_key_iv: "bf6122c98ff09d216b9ed40f9a9d661a"
server_key_version: "encryptionkey@0"
I20260808 17:01:00.960572 4205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:00.979233 4205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:00.979506 4205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:00.979645 4205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:00.979846 4205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:00.980149 4205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:00.980203 4205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:00.980242 4205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:00.980269 4205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:00.986129 4205 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:45389
I20260808 17:01:00.986181 4375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:45389 every 8 connection(s)
I20260808 17:01:00.986491 4205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 17:01:00.991612 3814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.987202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:55851 (local address 127.8.14.126:35175)
0808 17:01:00.987283 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:00.987285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:00.987494 (+ 209us) server_negotiation.cc:407] Connection header received
0808 17:01:00.988013 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:00.988014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:00.988027 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:00.988076 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:00.988756 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.989258 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.990241 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.990451 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.990531 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.991292 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:00.991295 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:00.991295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:00.991306 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:00.991322 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:00.991326 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:00.991398 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:00.991510 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 17:01:00.991537 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 17:01:00.991822 4379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.987161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:35175 (local address 127.8.14.67:55851)
0808 17:01:00.987375 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:00.987406 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:00.987903 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:00.988157 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:00.988163 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:00.988177 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:00.988658 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.988662 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.989375 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.989376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.990139 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:00.990141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.990539 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.990541 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:00.990675 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.991150 (+ 475us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:01:00.991150 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:00.991177 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:01:00.991199 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:00.991398 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:00.991404 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:00.991408 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:01:00.991499 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 17:01:00.991594 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260808 17:01:00.992331 4377 heartbeater.cc:347] Connected to a master server at 127.8.14.126:35175
I20260808 17:01:00.992408 4377 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:00.992619 4377 heartbeater.cc:511] Master 127.8.14.126:35175 requested a full tablet report, sending...
I20260808 17:01:00.992957 3759 ts_manager.cc:194] Registered new tserver with Master: 691f0c68105944ab9a778857533442e6 (127.8.14.67:45389)
I20260808 17:01:00.993330 3759 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.14.67:55851
I20260808 17:01:00.993634 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 4205
I20260808 17:01:00.993739 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 17:01:00.993928 8249 external_mini_cluster.cc:1428] Setting key a7842c16a1ffa4edd6cec2126b39e122
I20260808 17:01:00.994796 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:01:00.999264 3814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:00.996253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35986 (local address 127.8.14.126:35175)
0808 17:01:00.996289 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:00.996291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:00.996448 (+ 157us) server_negotiation.cc:407] Connection header received
0808 17:01:00.996526 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:00.996527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:00.996537 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:00.996586 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:00.996748 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.997196 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.997955 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:00.998147 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:00.998223 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:00.998976 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:00.998977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:00.998978 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:00.998982 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:00.998995 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:00.998999 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:00.999107 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:00.999175 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 17:01:00.999185 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:01:01.000679 3759 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:01:01.002126 3759 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:01:01.008215 4390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:01.004270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:37643 (local address 127.0.0.1:59742)
0808 17:01:01.004745 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:01.004755 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:01.004791 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:01.005110 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:01.005110 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:01.005113 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:01.005155 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:01.005157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.006355 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.006356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:01.007257 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:01.007260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.007931 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.007932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:01.008005 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:01.008018 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 17:01:01.008094 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 17:01:01.008144 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":85,"threads_started":1}
I20260808 17:01:01.008226 4395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:01.004605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59742 (local address 127.8.14.65:37643)
0808 17:01:01.004807 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:01.004812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:01.004823 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:01:01.004874 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:01.004876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:01.004890 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:01.005035 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:01.005235 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.006272 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.007342 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.007828 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.007928 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:01.008020 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:01.008109 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 17:01:01.008144 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":88,"threads_started":1}
I20260808 17:01:01.008740 4394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:01.004409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48930 (local address 127.8.14.66:38761)
0808 17:01:01.004572 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:01.004577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:01.004588 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:01:01.004630 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:01.004632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:01.004644 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:01.004752 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:01.004920 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.006052 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.007409 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.008128 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.008200 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:01.008425 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:01.008615 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 17:01:01.008657 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
I20260808 17:01:01.008764 4391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:01.004246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:38761 (local address 127.0.0.1:48930)
0808 17:01:01.004414 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:01.004453 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:01.004520 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:01.004761 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:01.004764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:01.004766 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:01.004844 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:01.004845 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.006137 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.006138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:01.007302 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:01.007305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.008462 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.008464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:01.008530 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:01.008533 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:01.008597 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 17:01:01.008646 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":13,"thread_start_us":60,"threads_started":1}
I20260808 17:01:01.008764 4389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:01.004246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:45389 (local address 127.0.0.1:43170)
0808 17:01:01.004618 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:01.004628 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:01.004672 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:01.005085 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:01.005086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:01.005088 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:01.005125 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:01.005126 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.005989 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.005990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:01.007307 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:01.007310 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.008484 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.008485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:01.008535 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:01.008538 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:01.008597 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 17:01:01.008646 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":16,"thread_start_us":52,"threads_started":1}
I20260808 17:01:01.009018 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:01.004415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43170 (local address 127.8.14.67:45389)
0808 17:01:01.004882 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:01.004886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:01.004897 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:01:01.004932 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:01.004934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:01.004948 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:01.005023 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:01.005199 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.005888 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.007409 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.008128 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.008200 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:01.008281 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:01.008885 (+ 604us) server_negotiation.cc:299] Negotiation successful
0808 17:01:01.008929 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":82,"threads_started":1}
I20260808 17:01:01.008970 3936 tablet_service.cc:1467] Processing CreateTablet for tablet 40efe920a24646c58eec09401d5d949d (DEFAULT_TABLE table=test-table [id=5c0a2b211c8e41a3897c966422a89e44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:01.009377 3936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40efe920a24646c58eec09401d5d949d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:01.009661 4288 tablet_service.cc:1467] Processing CreateTablet for tablet 40efe920a24646c58eec09401d5d949d (DEFAULT_TABLE table=test-table [id=5c0a2b211c8e41a3897c966422a89e44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:01.009980 4288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40efe920a24646c58eec09401d5d949d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:01.011670 4114 tablet_service.cc:1467] Processing CreateTablet for tablet 40efe920a24646c58eec09401d5d949d (DEFAULT_TABLE table=test-table [id=5c0a2b211c8e41a3897c966422a89e44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:01.012009 4114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40efe920a24646c58eec09401d5d949d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:01.012600 4396 tablet_bootstrap.cc:492] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b: Bootstrap starting.
I20260808 17:01:01.013110 4397 tablet_bootstrap.cc:492] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6: Bootstrap starting.
I20260808 17:01:01.013270 4396 tablet_bootstrap.cc:654] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:01.013619 4396 log.cc:824] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:01.013775 4397 tablet_bootstrap.cc:654] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:01.014107 4397 log.cc:824] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:01.014441 4396 tablet_bootstrap.cc:492] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b: No bootstrap required, opened a new log
I20260808 17:01:01.014525 4396 ts_tablet_manager.cc:1397] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:01:01.014745 4397 tablet_bootstrap.cc:492] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6: No bootstrap required, opened a new log
I20260808 17:01:01.014788 4397 ts_tablet_manager.cc:1397] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:01:01.015403 4401 tablet_bootstrap.cc:492] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988: Bootstrap starting.
I20260808 17:01:01.016096 4401 tablet_bootstrap.cc:654] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:01.016405 4401 log.cc:824] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:01.016770 4396 raft_consensus.cc:348] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691f0c68105944ab9a778857533442e6" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45389 } } peers { permanent_uuid: "eaba7dc432c445ac84e923b217a01988" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38761 } } peers { permanent_uuid: "1c05eebc5600425098470f97101de51b" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 37643 } }
I20260808 17:01:01.016901 4396 raft_consensus.cc:374] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:01.016922 4396 raft_consensus.cc:729] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c05eebc5600425098470f97101de51b, State: Initialized, Role: FOLLOWER
I20260808 17:01:01.017006 4396 consensus_queue.cc:260] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691f0c68105944ab9a778857533442e6" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45389 } } peers { permanent_uuid: "eaba7dc432c445ac84e923b217a01988" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38761 } } peers { permanent_uuid: "1c05eebc5600425098470f97101de51b" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 37643 } }
I20260808 17:01:01.017194 4396 ts_tablet_manager.cc:1428] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:01:01.017318 4025 heartbeater.cc:503] Master 127.8.14.126:35175 was elected leader, sending a full tablet report...
I20260808 17:01:01.017553 4397 raft_consensus.cc:348] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691f0c68105944ab9a778857533442e6" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45389 } } peers { permanent_uuid: "eaba7dc432c445ac84e923b217a01988" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38761 } } peers { permanent_uuid: "1c05eebc5600425098470f97101de51b" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 37643 } }
I20260808 17:01:01.017731 4397 raft_consensus.cc:374] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:01.017771 4397 raft_consensus.cc:729] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 691f0c68105944ab9a778857533442e6, State: Initialized, Role: FOLLOWER
I20260808 17:01:01.017886 4397 consensus_queue.cc:260] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691f0c68105944ab9a778857533442e6" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45389 } } peers { permanent_uuid: "eaba7dc432c445ac84e923b217a01988" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38761 } } peers { permanent_uuid: "1c05eebc5600425098470f97101de51b" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 37643 } }
I20260808 17:01:01.018167 4397 ts_tablet_manager.cc:1428] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:01:01.018193 4377 heartbeater.cc:503] Master 127.8.14.126:35175 was elected leader, sending a full tablet report...
I20260808 17:01:01.018823 4401 tablet_bootstrap.cc:492] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988: No bootstrap required, opened a new log
I20260808 17:01:01.018918 4401 ts_tablet_manager.cc:1397] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 17:01:01.020844 4401 raft_consensus.cc:348] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691f0c68105944ab9a778857533442e6" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45389 } } peers { permanent_uuid: "eaba7dc432c445ac84e923b217a01988" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38761 } } peers { permanent_uuid: "1c05eebc5600425098470f97101de51b" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 37643 } }
I20260808 17:01:01.020963 4401 raft_consensus.cc:374] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:01.020984 4401 raft_consensus.cc:729] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaba7dc432c445ac84e923b217a01988, State: Initialized, Role: FOLLOWER
I20260808 17:01:01.021114 4401 consensus_queue.cc:260] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691f0c68105944ab9a778857533442e6" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45389 } } peers { permanent_uuid: "eaba7dc432c445ac84e923b217a01988" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38761 } } peers { permanent_uuid: "1c05eebc5600425098470f97101de51b" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 37643 } }
I20260808 17:01:01.021380 4401 ts_tablet_manager.cc:1428] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 17:01:01.021436 4202 heartbeater.cc:503] Master 127.8.14.126:35175 was elected leader, sending a full tablet report...
W20260808 17:01:01.102653 4203 tablet.cc:2369] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 17:01:01.221737 4026 tablet.cc:2369] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:01:01.230971 4410 raft_consensus.cc:482] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:01:01.231110 4410 raft_consensus.cc:504] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691f0c68105944ab9a778857533442e6" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45389 } } peers { permanent_uuid: "eaba7dc432c445ac84e923b217a01988" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38761 } } peers { permanent_uuid: "1c05eebc5600425098470f97101de51b" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 37643 } }
I20260808 17:01:01.231506 4410 leader_election.cc:288] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 691f0c68105944ab9a778857533442e6 (127.8.14.67:45389), 1c05eebc5600425098470f97101de51b (127.8.14.65:37643)
I20260808 17:01:01.234472 4411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:01.231716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:37643 (local address 127.8.14.66:59961)
0808 17:01:01.231891 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:01.231900 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:01.231954 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:01.232093 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:01.232093 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:01.232095 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:01.232198 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:01.232200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.232829 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.232830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:01.233655 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:01.233658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.234191 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.234192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:01.234277 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:01.234281 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:01:01.234325 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 17:01:01.234379 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":3,"thread_start_us":112,"threads_started":1}
I20260808 17:01:01.234469 4395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:01.231766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:59961 (local address 127.8.14.65:37643)
0808 17:01:01.231841 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:01.231844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:01.231959 (+ 115us) server_negotiation.cc:407] Connection header received
0808 17:01:01.232001 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:01.232002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:01.232012 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:01.232090 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:01.232277 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.232743 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.233776 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.234107 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.234164 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:01.234193 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:01.234350 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 17:01:01.234396 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 17:01:01.234522 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:01.231919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:54551 (local address 127.8.14.67:45389)
0808 17:01:01.231971 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:01.231973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:01.231982 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:01:01.232012 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:01.232013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:01.232023 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:01.232075 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:01.232285 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.232743 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.233746 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.234118 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.234206 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:01.234215 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:01.234406 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 17:01:01.234448 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:01:01.234565 4204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:01.231665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:45389 (local address 127.8.14.66:54551)
0808 17:01:01.231791 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:01.231804 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:01.231862 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:01.232103 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:01.232105 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:01.232106 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:01.232199 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:01.232200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.232836 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.232838 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:01.233644 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:01.233646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.234241 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.234242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:01.234302 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:01.234305 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:01.234383 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 17:01:01.234422 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":42}
I20260808 17:01:01.234942 3961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40efe920a24646c58eec09401d5d949d" candidate_uuid: "eaba7dc432c445ac84e923b217a01988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c05eebc5600425098470f97101de51b" is_pre_election: true
I20260808 17:01:01.234966 4317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40efe920a24646c58eec09401d5d949d" candidate_uuid: "eaba7dc432c445ac84e923b217a01988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691f0c68105944ab9a778857533442e6" is_pre_election: true
I20260808 17:01:01.235095 3961 raft_consensus.cc:2454] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaba7dc432c445ac84e923b217a01988 in term 0.
I20260808 17:01:01.235140 4317 raft_consensus.cc:2454] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaba7dc432c445ac84e923b217a01988 in term 0.
I20260808 17:01:01.235349 4055 leader_election.cc:302] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c05eebc5600425098470f97101de51b, eaba7dc432c445ac84e923b217a01988; no voters:
I20260808 17:01:01.235479 4410 raft_consensus.cc:2781] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:01:01.235536 4410 raft_consensus.cc:482] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:01:01.235559 4410 raft_consensus.cc:3037] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:01.236274 4410 raft_consensus.cc:504] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691f0c68105944ab9a778857533442e6" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45389 } } peers { permanent_uuid: "eaba7dc432c445ac84e923b217a01988" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38761 } } peers { permanent_uuid: "1c05eebc5600425098470f97101de51b" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 37643 } }
I20260808 17:01:01.236626 3961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40efe920a24646c58eec09401d5d949d" candidate_uuid: "eaba7dc432c445ac84e923b217a01988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c05eebc5600425098470f97101de51b"
I20260808 17:01:01.236701 3961 raft_consensus.cc:3037] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b [term 0 FOLLOWER]: Advancing to term 1
W20260808 17:01:01.237010 4378 tablet.cc:2369] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:01:01.237246 4317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40efe920a24646c58eec09401d5d949d" candidate_uuid: "eaba7dc432c445ac84e923b217a01988" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691f0c68105944ab9a778857533442e6"
I20260808 17:01:01.237334 4317 raft_consensus.cc:3037] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:01.237658 3961 raft_consensus.cc:2454] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaba7dc432c445ac84e923b217a01988 in term 1.
I20260808 17:01:01.236534 4410 leader_election.cc:288] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [CANDIDATE]: Term 1 election: Requested vote from peers 691f0c68105944ab9a778857533442e6 (127.8.14.67:45389), 1c05eebc5600425098470f97101de51b (127.8.14.65:37643)
I20260808 17:01:01.237849 4055 leader_election.cc:302] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c05eebc5600425098470f97101de51b, eaba7dc432c445ac84e923b217a01988; no voters:
I20260808 17:01:01.237943 4410 raft_consensus.cc:2781] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:01.238125 4410 raft_consensus.cc:686] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [term 1 LEADER]: Becoming Leader. State: Replica: eaba7dc432c445ac84e923b217a01988, State: Running, Role: LEADER
I20260808 17:01:01.238229 4410 consensus_queue.cc:237] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691f0c68105944ab9a778857533442e6" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45389 } } peers { permanent_uuid: "eaba7dc432c445ac84e923b217a01988" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38761 } } peers { permanent_uuid: "1c05eebc5600425098470f97101de51b" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 37643 } }
I20260808 17:01:01.238309 4317 raft_consensus.cc:2454] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaba7dc432c445ac84e923b217a01988 in term 1.
I20260808 17:01:01.239084 3759 catalog_manager.cc:5495] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 reported cstate change: term changed from 0 to 1, leader changed from <none> to eaba7dc432c445ac84e923b217a01988 (127.8.14.66). New cstate: current_term: 1 leader_uuid: "eaba7dc432c445ac84e923b217a01988" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691f0c68105944ab9a778857533442e6" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaba7dc432c445ac84e923b217a01988" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 38761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c05eebc5600425098470f97101de51b" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 37643 } health_report { overall_health: UNKNOWN } } }
I20260808 17:01:01.290076 4394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:01.287997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48936 (local address 127.8.14.66:38761)
0808 17:01:01.288057 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:01.288059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:01.288075 (+ 16us) server_negotiation.cc:407] Connection header received
0808 17:01:01.288111 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:01.288112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:01.288131 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:01.288198 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:01.288325 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.288778 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.289464 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:01.289640 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:01.289689 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:01.289807 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:01.289892 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:01.289962 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:01.290009 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 17:01:01.290016 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 17:01:01.291971 4317 raft_consensus.cc:1261] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6 [term 1 FOLLOWER]: Refusing update from remote peer eaba7dc432c445ac84e923b217a01988: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:01.291972 3961 raft_consensus.cc:1261] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b [term 1 FOLLOWER]: Refusing update from remote peer eaba7dc432c445ac84e923b217a01988: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:01.292412 4410 consensus_queue.cc:1035] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c05eebc5600425098470f97101de51b" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 37643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:01.292596 4412 consensus_queue.cc:1035] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [LEADER]: Connected to new peer: Peer: permanent_uuid: "691f0c68105944ab9a778857533442e6" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 45389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:01.295894 4418 mvcc.cc:205] Tried to move back new op lower bound from 7316309857449254912 to 7316309857232408576. Current Snapshot: MvccSnapshot[applied={T|T < 7316309857449254912}]
I20260808 17:01:01.296343 4416 mvcc.cc:205] Tried to move back new op lower bound from 7316309857449254912 to 7316309857232408576. Current Snapshot: MvccSnapshot[applied={T|T < 7316309857449254912 or (T in {7316309857449254912})}]
I20260808 17:01:01.297396 3759 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:01.297495 3759 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:01.298308 3759 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5edb265373a2467d83780b2850de4371: Sending DeleteTablet for 3 replicas of tablet 40efe920a24646c58eec09401d5d949d
I20260808 17:01:01.298632 3936 tablet_service.cc:1514] Processing DeleteTablet for tablet 40efe920a24646c58eec09401d5d949d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:01 UTC) from {username='slave'} at 127.0.0.1:59742
I20260808 17:01:01.298759 4288 tablet_service.cc:1514] Processing DeleteTablet for tablet 40efe920a24646c58eec09401d5d949d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:01 UTC) from {username='slave'} at 127.0.0.1:43170
I20260808 17:01:01.298846 4427 tablet_replica.cc:330] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b: stopping tablet replica
I20260808 17:01:01.298981 4427 raft_consensus.cc:2229] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:01.299101 4427 raft_consensus.cc:2258] T 40efe920a24646c58eec09401d5d949d P 1c05eebc5600425098470f97101de51b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:01.299095 4114 tablet_service.cc:1514] Processing DeleteTablet for tablet 40efe920a24646c58eec09401d5d949d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:01 UTC) from {username='slave'} at 127.0.0.1:48930
I20260808 17:01:01.299201 4428 tablet_replica.cc:330] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6: stopping tablet replica
I20260808 17:01:01.299290 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 3843
I20260808 17:01:01.299311 4428 raft_consensus.cc:2229] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:01.299428 4428 raft_consensus.cc:2258] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:01.300087 4428 ts_tablet_manager.cc:1905] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:01.301528 4428 ts_tablet_manager.cc:1918] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:01:01.301613 4428 log.cc:1196] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/40efe920a24646c58eec09401d5d949d
I20260808 17:01:01.301853 4428 ts_tablet_manager.cc:1939] T 40efe920a24646c58eec09401d5d949d P 691f0c68105944ab9a778857533442e6: Deleting consensus metadata
I20260808 17:01:01.302253 3738 catalog_manager.cc:4841] TS 691f0c68105944ab9a778857533442e6 (127.8.14.67:45389): tablet 40efe920a24646c58eec09401d5d949d (table test-table [id=5c0a2b211c8e41a3897c966422a89e44]) successfully deleted
I20260808 17:01:01.303016 4430 tablet_replica.cc:330] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988: stopping tablet replica
I20260808 17:01:01.303118 4430 raft_consensus.cc:2229] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:01:01.303253 4430 raft_consensus.cc:2258] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:01.303604 4430 ts_tablet_manager.cc:1905] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:01.305178 4430 ts_tablet_manager.cc:1918] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:01:01.305263 4430 log.cc:1196] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/40efe920a24646c58eec09401d5d949d
I20260808 17:01:01.305483 4430 ts_tablet_manager.cc:1939] T 40efe920a24646c58eec09401d5d949d P eaba7dc432c445ac84e923b217a01988: Deleting consensus metadata
I20260808 17:01:01.305840 3737 catalog_manager.cc:4841] TS eaba7dc432c445ac84e923b217a01988 (127.8.14.66:38761): tablet 40efe920a24646c58eec09401d5d949d (table test-table [id=5c0a2b211c8e41a3897c966422a89e44]) successfully deleted
W20260808 17:01:01.317245 3739 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.14.65:37643 (error 108)
I20260808 17:01:01.317793 4389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:01.317640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:37643 (local address 127.0.0.1:59744)
0808 17:01:01.317709 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:01.317753 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.65:37643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 17:01:01.317891 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 4028
W20260808 17:01:01.317915 3739 catalog_manager.cc:4584] TS 1c05eebc5600425098470f97101de51b (127.8.14.65:37643): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 40efe920a24646c58eec09401d5d949d: Network error: Client connection negotiation failed: client connection to 127.8.14.65:37643: connect: Connection refused (error 111)
I20260808 17:01:01.328699 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 4205
I20260808 17:01:01.336756 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 3717
2026-08-08T17:01:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (976 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:01 dist-test-slave-1nl6 krb5kdc[4437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:01 dist-test-slave-1nl6 krb5kdc[4437](info): set up 2 sockets
Aug 08 17:01:01 dist-test-slave-1nl6 krb5kdc[4437](info): commencing operation
krb5kdc: starting...
W20260808 17:01:02.570793 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:02 dist-test-slave-1nl6 krb5kdc[4437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 17:01:02.691322 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:38703
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:45637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:38703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:02.772004 4456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:02.772266 4456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:02.772315 4456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:02.773440 4456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:02.773511 4456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:02.773537 4456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:02.773558 4456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:02.773586 4456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:02.775060 4456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:45637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:38703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:38703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:02.775354 4456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:02.775583 4456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:02.780380 4466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:02.780488 4468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:02.780503 4456 server_base.cc:1034] running on GCE node
W20260808 17:01:02.780390 4464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:02.781001 4456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:02.781314 4456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:02.782482 4456 hybrid_clock.cc:648] HybridClock initialized: now 1786208462782467 us; error 28 us; skew 500 ppm
Aug 08 17:01:02 dist-test-slave-1nl6 krb5kdc[4437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208462, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:02.785223 4456 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 17:01:02.785616 4456 webserver.cc:460] Webserver started at http://127.8.14.126:44423/ using document root <none> and password file <none>
I20260808 17:01:02.785791 4456 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:02.785831 4456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:02.785928 4456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:02.786890 4456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "4352a403debd4a1fb60d1270c64aeb16"
format_stamp: "Formatted at 2026-08-08 17:01:02 on dist-test-slave-1nl6"
server_key: "7db60d36174c2f55921ce914b128758d"
server_key_iv: "e8f6cc6b8ae33274128e1083e792e909"
server_key_version: "encryptionkey@0"
I20260808 17:01:02.787281 4456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "4352a403debd4a1fb60d1270c64aeb16"
format_stamp: "Formatted at 2026-08-08 17:01:02 on dist-test-slave-1nl6"
server_key: "7db60d36174c2f55921ce914b128758d"
server_key_iv: "e8f6cc6b8ae33274128e1083e792e909"
server_key_version: "encryptionkey@0"
I20260808 17:01:02.788604 4456 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:02.789355 4475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:02.789535 4456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:02.789631 4456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "4352a403debd4a1fb60d1270c64aeb16"
format_stamp: "Formatted at 2026-08-08 17:01:02 on dist-test-slave-1nl6"
server_key: "7db60d36174c2f55921ce914b128758d"
server_key_iv: "e8f6cc6b8ae33274128e1083e792e909"
server_key_version: "encryptionkey@0"
I20260808 17:01:02.789705 4456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:02.812233 4456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:02.814584 4456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:02.814698 4456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:02.818699 4456 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:38703
I20260808 17:01:02.818760 4551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:38703 every 8 connection(s)
I20260808 17:01:02.819133 4456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:02.820014 4552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:02.822378 4552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16: Bootstrap starting.
I20260808 17:01:02.823005 4552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:02.823305 4552 log.cc:824] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:02.824019 4552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16: No bootstrap required, opened a new log
I20260808 17:01:02.825632 4552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4352a403debd4a1fb60d1270c64aeb16" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 38703 } }
I20260808 17:01:02.825769 4552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:02.825843 4552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4352a403debd4a1fb60d1270c64aeb16, State: Initialized, Role: FOLLOWER
I20260808 17:01:02.826014 4552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4352a403debd4a1fb60d1270c64aeb16" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 38703 } }
I20260808 17:01:02.826104 4552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:02.826179 4552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:02.826241 4552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:02.826354 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 4456
I20260808 17:01:02.826525 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:02.826710 8249 external_mini_cluster.cc:1428] Setting key 579c271c3d66057fb836c33e9b025fa7
I20260808 17:01:02.827026 4552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4352a403debd4a1fb60d1270c64aeb16" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 38703 } }
I20260808 17:01:02.827162 4552 leader_election.cc:302] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4352a403debd4a1fb60d1270c64aeb16; no voters:
I20260808 17:01:02.827375 4552 leader_election.cc:288] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:02.827402 4557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:02.827558 4557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [term 1 LEADER]: Becoming Leader. State: Replica: 4352a403debd4a1fb60d1270c64aeb16, State: Running, Role: LEADER
I20260808 17:01:02.827709 4552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:02.827698 4557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4352a403debd4a1fb60d1270c64aeb16" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 38703 } }
I20260808 17:01:02.828455 4560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4352a403debd4a1fb60d1270c64aeb16. Latest consensus state: current_term: 1 leader_uuid: "4352a403debd4a1fb60d1270c64aeb16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4352a403debd4a1fb60d1270c64aeb16" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 38703 } } }
I20260808 17:01:02.828543 4560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:02.829136 4559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4352a403debd4a1fb60d1270c64aeb16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4352a403debd4a1fb60d1270c64aeb16" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 38703 } } }
I20260808 17:01:02.829232 4559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:02.829461 4569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:02.829991 4569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:02.831687 4569 catalog_manager.cc:1347] Generated new cluster ID: 7ec2e014521146bfba37fba0ee93ae8c
I20260808 17:01:02.831753 4569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:01:02 dist-test-slave-1nl6 krb5kdc[4437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:01:02.839241 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:02.827178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43368 (local address 127.8.14.126:38703)
0808 17:01:02.827379 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:02.827389 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:02.827425 (+ 36us) server_negotiation.cc:407] Connection header received
0808 17:01:02.827544 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:02.827553 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:02.827627 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:02.828552 (+ 925us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:02.828955 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:02.830857 (+ 1902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:02.831816 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:02.832444 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:02.832541 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:02.835540 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:02.835545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:02.835545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:02.835557 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:02.837543 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:02.837963 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:02.837966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:02.837966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:02.838084 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:02.838329 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:02.838330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:02.838331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:02.838618 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:02.838770 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:02.838886 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 17:01:02.839035 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260808 17:01:02.844517 4569 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:02.845000 4569 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:02.849103 4569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16: Generated new TSK 0
I20260808 17:01:02.849273 4569 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.14.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
Entry for principal kudu/127.8.14.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab.
I20260808 17:01:02.881922 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:38703
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:45637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:02.965412 4583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:02.965636 4583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:02.965682 4583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:02.966795 4583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:02.966858 4583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:02.966899 4583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 17:01:02.968344 4583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:45637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.65.keytab
--principal=kudu/127.8.14.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:38703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:02.968596 4583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:02.968820 4583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:02.973805 4595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:02.973811 4591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:02.973814 4590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:02.974026 4583 server_base.cc:1034] running on GCE node
I20260808 17:01:02.974236 4583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:02.974515 4583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:02.975672 4583 hybrid_clock.cc:648] HybridClock initialized: now 1786208462975663 us; error 37 us; skew 500 ppm
Aug 08 17:01:02 dist-test-slave-1nl6 krb5kdc[4437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208462, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:02.978232 4583 init.cc:377] Logged in from keytab as kudu/127.8.14.65@KRBTEST.COM (short username kudu)
I20260808 17:01:02.978644 4583 webserver.cc:460] Webserver started at http://127.8.14.65:41457/ using document root <none> and password file <none>
I20260808 17:01:02.978871 4583 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:02.978962 4583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:02.979071 4583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:02.980038 4583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "3d9e8579d0ce437e87127489b4eae6dc"
format_stamp: "Formatted at 2026-08-08 17:01:02 on dist-test-slave-1nl6"
server_key: "3f88df8cb3de187f730db6bf4cb486c2"
server_key_iv: "23654f81982efa41986a4bef9e0f0154"
server_key_version: "encryptionkey@0"
I20260808 17:01:02.980337 4583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "3d9e8579d0ce437e87127489b4eae6dc"
format_stamp: "Formatted at 2026-08-08 17:01:02 on dist-test-slave-1nl6"
server_key: "3f88df8cb3de187f730db6bf4cb486c2"
server_key_iv: "23654f81982efa41986a4bef9e0f0154"
server_key_version: "encryptionkey@0"
I20260808 17:01:02.981564 4583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:01:02.982231 4608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:02.982414 4583 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:02.982506 4583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "3d9e8579d0ce437e87127489b4eae6dc"
format_stamp: "Formatted at 2026-08-08 17:01:02 on dist-test-slave-1nl6"
server_key: "3f88df8cb3de187f730db6bf4cb486c2"
server_key_iv: "23654f81982efa41986a4bef9e0f0154"
server_key_version: "encryptionkey@0"
I20260808 17:01:02.982579 4583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:02.996306 4583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:02.998139 4583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:02.998298 4583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:02.998524 4583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:02.998850 4583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:02.998885 4583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:02.999053 4583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:02.999099 4583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:03.004882 4583 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:43393
I20260808 17:01:03.004930 4769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:43393 every 8 connection(s)
I20260808 17:01:03.005239 4583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 17:01:03.007292 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 4583
I20260808 17:01:03.007416 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 17:01:03.007619 8249 external_mini_cluster.cc:1428] Setting key 15a2f5a699f4325559279c95669eace8
Aug 08 17:01:03 dist-test-slave-1nl6 krb5kdc[4437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208462, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.65@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:01:03.014765 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.005942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:58797 (local address 127.8.14.126:38703)
0808 17:01:03.006009 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:03.006011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:03.006183 (+ 172us) server_negotiation.cc:407] Connection header received
0808 17:01:03.006660 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:03.006661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:03.006670 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:03.006728 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:03.007773 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.008337 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.009306 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.009626 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.009703 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.011879 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:03.011881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:03.011881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:03.011889 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:03.013331 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:03.013754 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:03.013756 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:03.013756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:03.013819 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:03.014086 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:03.014087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:03.014087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:03.014206 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:03.014277 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:03.014564 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 17:01:03.014668 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 17:01:03.014947 4773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.005861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:38703 (local address 127.8.14.65:58797)
0808 17:01:03.006050 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:03.006093 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:03.006572 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:03.006779 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:03.006782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:03.007411 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:03.007666 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.007672 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.008436 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.008437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.009211 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.009214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.009714 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.009715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.009811 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.010391 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:03.010405 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:03.011773 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:03.013431 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:03.013439 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:03.013441 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:03.013667 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:03.013888 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:03.013889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:03.013889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:03.014006 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:03.014280 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:03.014288 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:03.014425 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 17:01:03.014526 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 17:01:03.014683 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260808 17:01:03.015465 4771 heartbeater.cc:347] Connected to a master server at 127.8.14.126:38703
I20260808 17:01:03.015558 4771 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:03.015750 4771 heartbeater.cc:511] Master 127.8.14.126:38703 requested a full tablet report, sending...
I20260808 17:01:03.016265 4502 ts_manager.cc:194] Registered new tserver with Master: 3d9e8579d0ce437e87127489b4eae6dc (127.8.14.65:43393)
WARNING: no policy specified for kudu/127.8.14.66@KRBTEST.COM; defaulting to no policy
I20260808 17:01:03.016914 4502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.65@KRBTEST.COM'} at 127.8.14.65:58797
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
Entry for principal kudu/127.8.14.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab.
I20260808 17:01:03.033722 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:38703
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:45637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:03.115072 4778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:03.115270 4778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:03.115302 4778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:03.116364 4778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:03.116400 4778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:03.116425 4778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 17:01:03.117794 4778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:45637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.66.keytab
--principal=kudu/127.8.14.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:38703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:03.117977 4778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:03.118221 4778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:03.123172 4791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:03.123179 4788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:03.123179 4786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:03.123420 4778 server_base.cc:1034] running on GCE node
I20260808 17:01:03.123625 4778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:03.123900 4778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:03.125066 4778 hybrid_clock.cc:648] HybridClock initialized: now 1786208463125042 us; error 37 us; skew 500 ppm
Aug 08 17:01:03 dist-test-slave-1nl6 krb5kdc[4437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208463, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:03.127750 4778 init.cc:377] Logged in from keytab as kudu/127.8.14.66@KRBTEST.COM (short username kudu)
I20260808 17:01:03.128381 4778 webserver.cc:460] Webserver started at http://127.8.14.66:43239/ using document root <none> and password file <none>
I20260808 17:01:03.128685 4778 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:03.128756 4778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:03.128901 4778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:03.130040 4778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "9ff0d69147f64c2990e6e6595e727288"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "cc1cf633130231c0f2766b3c04b1993f"
server_key_iv: "533c3be10f7a5025402718d0b80d3e27"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.130411 4778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "9ff0d69147f64c2990e6e6595e727288"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "cc1cf633130231c0f2766b3c04b1993f"
server_key_iv: "533c3be10f7a5025402718d0b80d3e27"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.131839 4778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:01:03.132601 4800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:03.132771 4778 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:01:03.132882 4778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "9ff0d69147f64c2990e6e6595e727288"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "cc1cf633130231c0f2766b3c04b1993f"
server_key_iv: "533c3be10f7a5025402718d0b80d3e27"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.132967 4778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:03.146987 4778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:03.149353 4778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:03.149516 4778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:03.149740 4778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:03.150045 4778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:03.150100 4778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:03.150141 4778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:03.150169 4778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:03.156068 4778 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:37089
I20260808 17:01:03.156111 4962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:37089 every 8 connection(s)
I20260808 17:01:03.156445 4778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 17:01:03.158794 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 4778
I20260808 17:01:03.158946 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 17:01:03.159142 8249 external_mini_cluster.cc:1428] Setting key e636dc1939281bead85c41162e9bb315
Aug 08 17:01:03 dist-test-slave-1nl6 krb5kdc[4437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208463, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.66@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:01:03.166980 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.157173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:42407 (local address 127.8.14.126:38703)
0808 17:01:03.157214 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:03.157216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:03.157415 (+ 199us) server_negotiation.cc:407] Connection header received
0808 17:01:03.157881 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:03.157882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:03.157891 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:03.157950 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:03.159023 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.159770 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.161271 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.161557 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.161617 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.163811 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:03.163813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:03.163813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:03.163821 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:03.165564 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:03.165953 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:03.165954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:03.165954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:03.166021 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:03.166251 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:03.166252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:03.166253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:03.166403 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:03.166502 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:03.166737 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 17:01:03.166828 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:01:03.167018 4966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.157133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:38703 (local address 127.8.14.66:42407)
0808 17:01:03.157291 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:03.157332 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:03.157794 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:03.158000 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:03.158003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:03.158534 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:03.158841 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.158848 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.159881 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.159883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.161171 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.161174 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.161650 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.161651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.161811 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.162354 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:03.162362 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:03.163719 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:03.165658 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:03.165665 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:03.165667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:03.165877 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:03.166094 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:03.166095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:03.166095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:03.166185 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:03.166494 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:03.166499 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:03.166606 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 17:01:03.166685 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 17:01:03.166800 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260808 17:01:03.167550 4963 heartbeater.cc:347] Connected to a master server at 127.8.14.126:38703
I20260808 17:01:03.167639 4963 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:03.167840 4963 heartbeater.cc:511] Master 127.8.14.126:38703 requested a full tablet report, sending...
I20260808 17:01:03.168262 4502 ts_manager.cc:194] Registered new tserver with Master: 9ff0d69147f64c2990e6e6595e727288 (127.8.14.66:37089)
I20260808 17:01:03.168778 4502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.66@KRBTEST.COM'} at 127.8.14.66:42407
WARNING: no policy specified for kudu/127.8.14.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
Entry for principal kudu/127.8.14.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab.
I20260808 17:01:03.185146 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:38703
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:45637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:03.266742 4970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:03.267012 4970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:03.267042 4970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:03.268138 4970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:03.268208 4970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:03.268234 4970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 17:01:03.269727 4970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:45637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.67.keytab
--principal=kudu/127.8.14.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:38703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:03.269960 4970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:03.270175 4970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:01:03.275122 4970 server_base.cc:1034] running on GCE node
W20260808 17:01:03.275097 4981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:03.275116 4985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:03.275249 4980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:03.275492 4970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:03.275735 4970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:03.276878 4970 hybrid_clock.cc:648] HybridClock initialized: now 1786208463276870 us; error 29 us; skew 500 ppm
Aug 08 17:01:03 dist-test-slave-1nl6 krb5kdc[4437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208463, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:03.279366 4970 init.cc:377] Logged in from keytab as kudu/127.8.14.67@KRBTEST.COM (short username kudu)
I20260808 17:01:03.279857 4970 webserver.cc:460] Webserver started at http://127.8.14.67:45587/ using document root <none> and password file <none>
I20260808 17:01:03.280089 4970 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:03.280158 4970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:03.280292 4970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:03.281294 4970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "39fde207d91ab76ec4f86a5df9ece3bb"
server_key_iv: "0bb062fd84518f023b166e1f687d1b23"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.281662 4970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "39fde207d91ab76ec4f86a5df9ece3bb"
server_key_iv: "0bb062fd84518f023b166e1f687d1b23"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.283017 4970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:01:03.283720 4995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:03.283911 4970 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:03.284010 4970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "39fde207d91ab76ec4f86a5df9ece3bb"
server_key_iv: "0bb062fd84518f023b166e1f687d1b23"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.284098 4970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:03.293217 4970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:03.295473 4970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:03.295637 4970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:03.295869 4970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:03.296177 4970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:03.296238 4970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:03.296280 4970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:03.296310 4970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:03.302019 4970 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:40553
I20260808 17:01:03.302058 5144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:40553 every 8 connection(s)
I20260808 17:01:03.302368 4970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
Aug 08 17:01:03 dist-test-slave-1nl6 krb5kdc[4437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208463, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.67@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:01:03.310245 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 4970
I20260808 17:01:03.310340 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 17:01:03.310528 8249 external_mini_cluster.cc:1428] Setting key 13d7c82df3309d44eed24077d3c6c991
I20260808 17:01:03.311480 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.303108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:47637 (local address 127.8.14.126:38703)
0808 17:01:03.303181 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:03.303183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:03.303391 (+ 208us) server_negotiation.cc:407] Connection header received
0808 17:01:03.303904 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:03.303905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:03.303919 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:03.303983 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:03.304901 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.305416 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.306404 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.306619 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.306707 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.308796 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:03.308797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:03.308798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:03.308806 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:03.309981 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:03.310434 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:03.310436 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:03.310436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:03.310497 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:03.310778 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:03.310779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:03.310779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:03.310951 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:03.311018 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:03.311296 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 17:01:03.311392 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 17:01:03.311642 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.303093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:38703 (local address 127.8.14.67:47637)
0808 17:01:03.303271 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:03.303308 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:03.303792 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:03.304057 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:03.304060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:03.304608 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:03.304789 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.304793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.305529 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.305530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.306309 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.306311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.306720 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.306721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.306846 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.307361 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:03.307369 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:03.308689 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:03.310066 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:03.310074 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:03.310076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:03.310341 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:03.310577 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:03.310578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:03.310579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:03.310708 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:03.311008 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:03.311015 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:03.311153 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 17:01:03.311258 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 17:01:03.311415 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260808 17:01:03.312067 5145 heartbeater.cc:347] Connected to a master server at 127.8.14.126:38703
I20260808 17:01:03.312173 5145 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:03.312356 5145 heartbeater.cc:511] Master 127.8.14.126:38703 requested a full tablet report, sending...
I20260808 17:01:03.312727 4502 ts_manager.cc:194] Registered new tserver with Master: 0a20f37db6ce45d4a13cf2d56e1bc28f (127.8.14.67:40553)
I20260808 17:01:03.313053 4502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.14.67@KRBTEST.COM'} at 127.8.14.67:47637
I20260808 17:01:03.322336 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:01:03.329632 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.323779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43374 (local address 127.8.14.126:38703)
0808 17:01:03.323858 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:03.323860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:03.323962 (+ 102us) server_negotiation.cc:407] Connection header received
0808 17:01:03.324002 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:03.324002 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:03.324011 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:03.324053 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:03.324595 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.325043 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.325697 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.325888 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.325936 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.327096 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:03.327098 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:03.327098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:03.327105 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:03.328384 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:03.328790 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:03.328793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:03.328794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:03.328865 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:03.329077 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:03.329078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:03.329078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:03.329219 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:03.329340 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:03.329449 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 17:01:03.329531 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 17:01:03.331346 4502 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:01:03.332909 4502 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:01:03.339416 5157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.334975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:43393 (local address 127.0.0.1:54112)
0808 17:01:03.335266 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:03.335322 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:03.335390 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:03.335692 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:03.335695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:03.335698 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:03.335766 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.335768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.336976 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.336977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.338334 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.338338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.339090 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.339091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.339176 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.339189 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 17:01:03.339263 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 17:01:03.339322 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":48,"thread_start_us":53,"threads_started":1}
I20260808 17:01:03.339649 5158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.335011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:37089 (local address 127.0.0.1:44040)
0808 17:01:03.335356 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:03.335366 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:03.335434 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:03.335690 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:03.335694 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:03.335696 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:03.335761 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.335763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.337087 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.337087 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.338338 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.338340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.339374 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.339374 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.339436 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.339440 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:01:03.339497 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 17:01:03.339543 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":6,"thread_start_us":55,"threads_started":1}
I20260808 17:01:03.339741 5163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.335174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44040 (local address 127.8.14.66:37089)
0808 17:01:03.335356 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:03.335361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:03.335441 (+ 80us) server_negotiation.cc:407] Connection header received
0808 17:01:03.335546 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:03.335548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:03.335565 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:03.335676 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:03.335851 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.336981 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.338421 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.339106 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.339201 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.339249 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:03.339618 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 17:01:03.339660 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260808 17:01:03.339741 5160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.335073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54112 (local address 127.8.14.65:43393)
0808 17:01:03.335498 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:03.335503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:03.335515 (+ 12us) server_negotiation.cc:407] Connection header received
0808 17:01:03.335558 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:03.335560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:03.335575 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:03.335917 (+ 342us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:03.335993 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.336877 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.338442 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.338982 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.339086 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.339148 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:03.339620 (+ 472us) server_negotiation.cc:299] Negotiation successful
0808 17:01:03.339663 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":89,"threads_started":1}
I20260808 17:01:03.339548 5162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.335175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:40553 (local address 127.0.0.1:36598)
0808 17:01:03.335511 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:03.335519 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:03.335596 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:03.335995 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:03.335996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:03.335998 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:03.336052 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.336054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.337290 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.337291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.338221 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.338224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.339235 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.339237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.339325 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.339335 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 17:01:03.339408 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 17:01:03.339453 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":53,"threads_started":1}
I20260808 17:01:03.340049 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.335265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36598 (local address 127.8.14.67:40553)
0808 17:01:03.335752 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:03.335756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:03.335765 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:01:03.335801 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:03.335803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:03.335820 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:03.335908 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:03.336142 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.337187 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.338326 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.339062 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.339137 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.339199 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:03.339426 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 17:01:03.339666 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":85,"threads_started":1}
I20260808 17:01:03.340412 4679 tablet_service.cc:1467] Processing CreateTablet for tablet 2e18035562fb46ed9ddd02e0e135ef11 (DEFAULT_TABLE table=test-table [id=cad223b964864a28a7259ab3e11f84b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:03.340405 4875 tablet_service.cc:1467] Processing CreateTablet for tablet 2e18035562fb46ed9ddd02e0e135ef11 (DEFAULT_TABLE table=test-table [id=cad223b964864a28a7259ab3e11f84b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:03.340776 4875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e18035562fb46ed9ddd02e0e135ef11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:03.340776 4679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e18035562fb46ed9ddd02e0e135ef11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:03.340803 5059 tablet_service.cc:1467] Processing CreateTablet for tablet 2e18035562fb46ed9ddd02e0e135ef11 (DEFAULT_TABLE table=test-table [id=cad223b964864a28a7259ab3e11f84b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:03.341207 5059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e18035562fb46ed9ddd02e0e135ef11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:03.343981 5165 tablet_bootstrap.cc:492] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f: Bootstrap starting.
I20260808 17:01:03.344265 5166 tablet_bootstrap.cc:492] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288: Bootstrap starting.
I20260808 17:01:03.344547 5167 tablet_bootstrap.cc:492] T 2e18035562fb46ed9ddd02e0e135ef11 P 3d9e8579d0ce437e87127489b4eae6dc: Bootstrap starting.
I20260808 17:01:03.344724 5165 tablet_bootstrap.cc:654] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:03.344987 5166 tablet_bootstrap.cc:654] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:03.345208 5167 tablet_bootstrap.cc:654] T 2e18035562fb46ed9ddd02e0e135ef11 P 3d9e8579d0ce437e87127489b4eae6dc: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:03.345330 5165 log.cc:824] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:03.345585 5166 log.cc:824] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:03.345793 5167 log.cc:824] T 2e18035562fb46ed9ddd02e0e135ef11 P 3d9e8579d0ce437e87127489b4eae6dc: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:03.346146 5165 tablet_bootstrap.cc:492] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f: No bootstrap required, opened a new log
I20260808 17:01:03.346204 5165 ts_tablet_manager.cc:1397] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:01:03.346359 5166 tablet_bootstrap.cc:492] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288: No bootstrap required, opened a new log
I20260808 17:01:03.346426 5166 ts_tablet_manager.cc:1397] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 17:01:03.347517 5167 tablet_bootstrap.cc:492] T 2e18035562fb46ed9ddd02e0e135ef11 P 3d9e8579d0ce437e87127489b4eae6dc: No bootstrap required, opened a new log
I20260808 17:01:03.347592 5167 ts_tablet_manager.cc:1397] T 2e18035562fb46ed9ddd02e0e135ef11 P 3d9e8579d0ce437e87127489b4eae6dc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 17:01:03.348067 5166 raft_consensus.cc:348] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9e8579d0ce437e87127489b4eae6dc" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43393 } } peers { permanent_uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40553 } } peers { permanent_uuid: "9ff0d69147f64c2990e6e6595e727288" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37089 } }
I20260808 17:01:03.348235 5166 raft_consensus.cc:374] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:03.348142 5165 raft_consensus.cc:348] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9e8579d0ce437e87127489b4eae6dc" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43393 } } peers { permanent_uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40553 } } peers { permanent_uuid: "9ff0d69147f64c2990e6e6595e727288" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37089 } }
I20260808 17:01:03.348282 5166 raft_consensus.cc:729] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ff0d69147f64c2990e6e6595e727288, State: Initialized, Role: FOLLOWER
I20260808 17:01:03.348310 5165 raft_consensus.cc:374] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:03.348357 5165 raft_consensus.cc:729] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a20f37db6ce45d4a13cf2d56e1bc28f, State: Initialized, Role: FOLLOWER
I20260808 17:01:03.348409 5166 consensus_queue.cc:260] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9e8579d0ce437e87127489b4eae6dc" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43393 } } peers { permanent_uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40553 } } peers { permanent_uuid: "9ff0d69147f64c2990e6e6595e727288" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37089 } }
I20260808 17:01:03.348472 5165 consensus_queue.cc:260] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9e8579d0ce437e87127489b4eae6dc" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43393 } } peers { permanent_uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40553 } } peers { permanent_uuid: "9ff0d69147f64c2990e6e6595e727288" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37089 } }
I20260808 17:01:03.348721 5166 ts_tablet_manager.cc:1428] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:01:03.348747 5165 ts_tablet_manager.cc:1428] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 17:01:03.348788 5145 heartbeater.cc:503] Master 127.8.14.126:38703 was elected leader, sending a full tablet report...
I20260808 17:01:03.348863 4963 heartbeater.cc:503] Master 127.8.14.126:38703 was elected leader, sending a full tablet report...
I20260808 17:01:03.349684 5167 raft_consensus.cc:348] T 2e18035562fb46ed9ddd02e0e135ef11 P 3d9e8579d0ce437e87127489b4eae6dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9e8579d0ce437e87127489b4eae6dc" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43393 } } peers { permanent_uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40553 } } peers { permanent_uuid: "9ff0d69147f64c2990e6e6595e727288" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37089 } }
I20260808 17:01:03.349879 5167 raft_consensus.cc:374] T 2e18035562fb46ed9ddd02e0e135ef11 P 3d9e8579d0ce437e87127489b4eae6dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:03.349941 5167 raft_consensus.cc:729] T 2e18035562fb46ed9ddd02e0e135ef11 P 3d9e8579d0ce437e87127489b4eae6dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d9e8579d0ce437e87127489b4eae6dc, State: Initialized, Role: FOLLOWER
I20260808 17:01:03.350054 5167 consensus_queue.cc:260] T 2e18035562fb46ed9ddd02e0e135ef11 P 3d9e8579d0ce437e87127489b4eae6dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9e8579d0ce437e87127489b4eae6dc" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43393 } } peers { permanent_uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40553 } } peers { permanent_uuid: "9ff0d69147f64c2990e6e6595e727288" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37089 } }
I20260808 17:01:03.350308 5167 ts_tablet_manager.cc:1428] T 2e18035562fb46ed9ddd02e0e135ef11 P 3d9e8579d0ce437e87127489b4eae6dc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:01:03.350375 4771 heartbeater.cc:503] Master 127.8.14.126:38703 was elected leader, sending a full tablet report...
I20260808 17:01:03.370499 5172 raft_consensus.cc:482] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:01:03.370602 5172 raft_consensus.cc:504] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9e8579d0ce437e87127489b4eae6dc" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43393 } } peers { permanent_uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40553 } } peers { permanent_uuid: "9ff0d69147f64c2990e6e6595e727288" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37089 } }
I20260808 17:01:03.370879 5172 leader_election.cc:288] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d9e8579d0ce437e87127489b4eae6dc (127.8.14.65:43393), 9ff0d69147f64c2990e6e6595e727288 (127.8.14.66:37089)
I20260808 17:01:03.373777 5160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.371144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:60267 (local address 127.8.14.65:43393)
0808 17:01:03.371181 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:03.371183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:03.371191 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:01:03.371217 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:03.371217 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:03.371229 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:03.371296 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:03.371463 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.371948 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.373016 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.373423 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.373476 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.373484 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:03.373663 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 17:01:03.373702 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:01:03.373775 5148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.371039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:43393 (local address 127.8.14.67:60267)
0808 17:01:03.371086 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:03.371097 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:03.371135 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:03.371318 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:03.371319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:03.371321 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:03.371395 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.371397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.372134 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.372135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.372926 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.372929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.373507 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.373508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.373590 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.373593 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:03.373645 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 17:01:03.373691 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 17:01:03.373885 5163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.371115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:34005 (local address 127.8.14.66:37089)
0808 17:01:03.371157 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:03.371159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:03.371310 (+ 151us) server_negotiation.cc:407] Connection header received
0808 17:01:03.371341 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:03.371342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:03.371353 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:03.371414 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:03.371593 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.372173 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.373106 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.373513 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.373585 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.373595 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:03.373768 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 17:01:03.373812 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:01:03.374141 5177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.371039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:37089 (local address 127.8.14.67:34005)
0808 17:01:03.371232 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:03.371242 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:03.371300 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:03.371441 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:03.371442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:03.371444 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:03.371512 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.371514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.372261 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.372262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.373021 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.373024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.373596 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.373597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.373687 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.373691 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 17:01:03.373749 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 17:01:03.373786 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":41,"thread_start_us":88,"threads_started":1}
I20260808 17:01:03.374192 4709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e18035562fb46ed9ddd02e0e135ef11" candidate_uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d9e8579d0ce437e87127489b4eae6dc" is_pre_election: true
I20260808 17:01:03.374370 4709 raft_consensus.cc:2454] T 2e18035562fb46ed9ddd02e0e135ef11 P 3d9e8579d0ce437e87127489b4eae6dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a20f37db6ce45d4a13cf2d56e1bc28f in term 0.
I20260808 17:01:03.374507 4902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e18035562fb46ed9ddd02e0e135ef11" candidate_uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ff0d69147f64c2990e6e6595e727288" is_pre_election: true
I20260808 17:01:03.374606 5001 leader_election.cc:302] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a20f37db6ce45d4a13cf2d56e1bc28f, 3d9e8579d0ce437e87127489b4eae6dc; no voters:
I20260808 17:01:03.374656 4902 raft_consensus.cc:2454] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a20f37db6ce45d4a13cf2d56e1bc28f in term 0.
I20260808 17:01:03.374760 5172 raft_consensus.cc:2781] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:01:03.374815 5172 raft_consensus.cc:482] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:01:03.374840 5172 raft_consensus.cc:3037] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:03.375531 5172 raft_consensus.cc:504] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9e8579d0ce437e87127489b4eae6dc" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43393 } } peers { permanent_uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40553 } } peers { permanent_uuid: "9ff0d69147f64c2990e6e6595e727288" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37089 } }
I20260808 17:01:03.375681 5172 leader_election.cc:288] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [CANDIDATE]: Term 1 election: Requested vote from peers 3d9e8579d0ce437e87127489b4eae6dc (127.8.14.65:43393), 9ff0d69147f64c2990e6e6595e727288 (127.8.14.66:37089)
I20260808 17:01:03.375825 4709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e18035562fb46ed9ddd02e0e135ef11" candidate_uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d9e8579d0ce437e87127489b4eae6dc"
I20260808 17:01:03.375901 4709 raft_consensus.cc:3037] T 2e18035562fb46ed9ddd02e0e135ef11 P 3d9e8579d0ce437e87127489b4eae6dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:03.375913 4902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e18035562fb46ed9ddd02e0e135ef11" candidate_uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ff0d69147f64c2990e6e6595e727288"
I20260808 17:01:03.375998 4902 raft_consensus.cc:3037] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:03.376519 4709 raft_consensus.cc:2454] T 2e18035562fb46ed9ddd02e0e135ef11 P 3d9e8579d0ce437e87127489b4eae6dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a20f37db6ce45d4a13cf2d56e1bc28f in term 1.
I20260808 17:01:03.376685 5001 leader_election.cc:302] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a20f37db6ce45d4a13cf2d56e1bc28f, 3d9e8579d0ce437e87127489b4eae6dc; no voters:
I20260808 17:01:03.376749 4902 raft_consensus.cc:2454] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a20f37db6ce45d4a13cf2d56e1bc28f in term 1.
I20260808 17:01:03.376781 5172 raft_consensus.cc:2781] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:03.376955 5172 raft_consensus.cc:686] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [term 1 LEADER]: Becoming Leader. State: Replica: 0a20f37db6ce45d4a13cf2d56e1bc28f, State: Running, Role: LEADER
I20260808 17:01:03.377069 5172 consensus_queue.cc:237] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9e8579d0ce437e87127489b4eae6dc" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43393 } } peers { permanent_uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40553 } } peers { permanent_uuid: "9ff0d69147f64c2990e6e6595e727288" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37089 } }
I20260808 17:01:03.377852 4502 catalog_manager.cc:5495] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a20f37db6ce45d4a13cf2d56e1bc28f (127.8.14.67). New cstate: current_term: 1 leader_uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9e8579d0ce437e87127489b4eae6dc" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a20f37db6ce45d4a13cf2d56e1bc28f" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 40553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ff0d69147f64c2990e6e6595e727288" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37089 } health_report { overall_health: UNKNOWN } } }
I20260808 17:01:03.383682 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.381654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36602 (local address 127.8.14.67:40553)
0808 17:01:03.381718 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:03.381720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:03.381728 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:01:03.381764 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:03.381765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:03.381779 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:03.381843 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:03.381969 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.382367 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.383076 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.383242 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.383292 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.383410 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:03.383488 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:03.383540 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:03.383571 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 17:01:03.383585 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 17:01:03.385387 4709 raft_consensus.cc:1261] T 2e18035562fb46ed9ddd02e0e135ef11 P 3d9e8579d0ce437e87127489b4eae6dc [term 1 FOLLOWER]: Refusing update from remote peer 0a20f37db6ce45d4a13cf2d56e1bc28f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:03.385465 4902 raft_consensus.cc:1261] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288 [term 1 FOLLOWER]: Refusing update from remote peer 0a20f37db6ce45d4a13cf2d56e1bc28f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:03.385839 5172 consensus_queue.cc:1035] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ff0d69147f64c2990e6e6595e727288" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 37089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:03.385969 5179 consensus_queue.cc:1035] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d9e8579d0ce437e87127489b4eae6dc" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 43393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:03.388334 5184 mvcc.cc:205] Tried to move back new op lower bound from 7316309866024108032 to 7316309865993220096. Current Snapshot: MvccSnapshot[applied={T|T < 7316309866024108032}]
I20260808 17:01:03.390789 4502 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:03.390911 4502 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:03.391639 4502 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4352a403debd4a1fb60d1270c64aeb16: Sending DeleteTablet for 3 replicas of tablet 2e18035562fb46ed9ddd02e0e135ef11
I20260808 17:01:03.391993 4679 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e18035562fb46ed9ddd02e0e135ef11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:03 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:54112
I20260808 17:01:03.391999 4875 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e18035562fb46ed9ddd02e0e135ef11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:03 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:44040
I20260808 17:01:03.392001 5059 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e18035562fb46ed9ddd02e0e135ef11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:03 UTC) from {username='kudu', principal='kudu/127.8.14.126@KRBTEST.COM'} at 127.0.0.1:36598
I20260808 17:01:03.392230 5199 tablet_replica.cc:330] T 2e18035562fb46ed9ddd02e0e135ef11 P 3d9e8579d0ce437e87127489b4eae6dc: stopping tablet replica
I20260808 17:01:03.392237 5204 tablet_replica.cc:330] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f: stopping tablet replica
I20260808 17:01:03.392326 5199 raft_consensus.cc:2229] T 2e18035562fb46ed9ddd02e0e135ef11 P 3d9e8579d0ce437e87127489b4eae6dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:03.392336 5204 raft_consensus.cc:2229] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:01:03.392431 5199 raft_consensus.cc:2258] T 2e18035562fb46ed9ddd02e0e135ef11 P 3d9e8579d0ce437e87127489b4eae6dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:03.392450 5204 raft_consensus.cc:2258] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:03.392608 5201 tablet_replica.cc:330] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288: stopping tablet replica
I20260808 17:01:03.392692 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 4583
I20260808 17:01:03.392889 5204 ts_tablet_manager.cc:1905] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:03.392740 5201 raft_consensus.cc:2229] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:03.393210 5201 raft_consensus.cc:2258] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:03.394769 5204 ts_tablet_manager.cc:1918] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:01:03.394837 5204 log.cc:1196] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/2e18035562fb46ed9ddd02e0e135ef11
I20260808 17:01:03.395114 5204 ts_tablet_manager.cc:1939] T 2e18035562fb46ed9ddd02e0e135ef11 P 0a20f37db6ce45d4a13cf2d56e1bc28f: Deleting consensus metadata
I20260808 17:01:03.395447 4482 catalog_manager.cc:4841] TS 0a20f37db6ce45d4a13cf2d56e1bc28f (127.8.14.67:40553): tablet 2e18035562fb46ed9ddd02e0e135ef11 (table test-table [id=cad223b964864a28a7259ab3e11f84b2]) successfully deleted
I20260808 17:01:03.395668 5201 ts_tablet_manager.cc:1905] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:03.396836 5201 ts_tablet_manager.cc:1918] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:01:03.396919 5201 log.cc:1196] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/2e18035562fb46ed9ddd02e0e135ef11
I20260808 17:01:03.397222 5201 ts_tablet_manager.cc:1939] T 2e18035562fb46ed9ddd02e0e135ef11 P 9ff0d69147f64c2990e6e6595e727288: Deleting consensus metadata
I20260808 17:01:03.397600 4478 catalog_manager.cc:4841] TS 9ff0d69147f64c2990e6e6595e727288 (127.8.14.66:37089): tablet 2e18035562fb46ed9ddd02e0e135ef11 (table test-table [id=cad223b964864a28a7259ab3e11f84b2]) successfully deleted
W20260808 17:01:03.403016 4482 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.14.65:43393 (error 108)
I20260808 17:01:03.403316 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 4778
I20260808 17:01:03.403465 5162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.403337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:43393 (local address 127.0.0.1:54122)
0808 17:01:03.403393 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:03.403430 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.65:43393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 17:01:03.406044 4482 catalog_manager.cc:4584] TS 3d9e8579d0ce437e87127489b4eae6dc (127.8.14.65:43393): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e18035562fb46ed9ddd02e0e135ef11: Network error: Client connection negotiation failed: client connection to 127.8.14.65:43393: connect: Connection refused (error 111)
I20260808 17:01:03.418758 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 4970
I20260808 17:01:03.429677 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 4456
2026-08-08T17:01:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2092 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T17:01:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:03Z Disabled control of system clock
I20260808 17:01:03.461980 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:32865
--webserver_interface=127.8.14.126
--webserver_port=0
--builtin_ntp_servers=127.8.14.84:41993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:32865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:01:03.543733 5217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:03.543956 5217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:03.543977 5217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:03.545125 5217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:03.545167 5217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:03.545181 5217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:03.545217 5217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:03.545243 5217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:03.546639 5217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:32865
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:32865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:03.546917 5217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:03.547216 5217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:03.551982 5230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:03.552007 5227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:03.552054 5217 server_base.cc:1034] running on GCE node
W20260808 17:01:03.551955 5223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:03.552361 5217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:03.552670 5217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:03.553823 5217 hybrid_clock.cc:648] HybridClock initialized: now 1786208463553803 us; error 37 us; skew 500 ppm
I20260808 17:01:03.555133 5217 webserver.cc:460] Webserver started at http://127.8.14.126:42347/ using document root <none> and password file <none>
I20260808 17:01:03.555373 5217 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:03.555423 5217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:03.555585 5217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:03.556658 5217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "7deed45832304812a05a289feeb6d22a"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "1462448cb9d7a4fe9a095e40122ac76c"
server_key_iv: "9d23bf72ce9cb51440d02955a8e26adb"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.557025 5217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "7deed45832304812a05a289feeb6d22a"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "1462448cb9d7a4fe9a095e40122ac76c"
server_key_iv: "9d23bf72ce9cb51440d02955a8e26adb"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.558322 5217 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:01:03.559052 5237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:03.559209 5217 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:03.559298 5217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "7deed45832304812a05a289feeb6d22a"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "1462448cb9d7a4fe9a095e40122ac76c"
server_key_iv: "9d23bf72ce9cb51440d02955a8e26adb"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.559355 5217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:03.573226 5217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:03.573464 5217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:03.573561 5217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:03.577625 5217 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:32865
I20260808 17:01:03.577675 5298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:32865 every 8 connection(s)
I20260808 17:01:03.577988 5217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:03.579012 5299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:03.581249 5299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a: Bootstrap starting.
I20260808 17:01:03.581816 5299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:03.582129 5299 log.cc:824] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:03.582787 5299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a: No bootstrap required, opened a new log
I20260808 17:01:03.584398 5299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7deed45832304812a05a289feeb6d22a" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 32865 } }
I20260808 17:01:03.584555 5299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:03.584605 5299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7deed45832304812a05a289feeb6d22a, State: Initialized, Role: FOLLOWER
I20260808 17:01:03.584748 5299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7deed45832304812a05a289feeb6d22a" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 32865 } }
I20260808 17:01:03.584827 5299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:03.584879 5299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:03.584928 5299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:03.585415 5299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7deed45832304812a05a289feeb6d22a" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 32865 } }
I20260808 17:01:03.585534 5299 leader_election.cc:302] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7deed45832304812a05a289feeb6d22a; no voters:
I20260808 17:01:03.585722 5299 leader_election.cc:288] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:03.585768 5302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:03.585978 5299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:03.586014 5302 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [term 1 LEADER]: Becoming Leader. State: Replica: 7deed45832304812a05a289feeb6d22a, State: Running, Role: LEADER
I20260808 17:01:03.586105 5302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7deed45832304812a05a289feeb6d22a" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 32865 } }
I20260808 17:01:03.586597 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 5217
I20260808 17:01:03.586717 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:03.586938 8249 external_mini_cluster.cc:1428] Setting key 3e486ea693fd8ed4b023746a3800ed46
I20260808 17:01:03.587189 5308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7deed45832304812a05a289feeb6d22a. Latest consensus state: current_term: 1 leader_uuid: "7deed45832304812a05a289feeb6d22a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7deed45832304812a05a289feeb6d22a" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 32865 } } }
I20260808 17:01:03.587258 5306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7deed45832304812a05a289feeb6d22a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7deed45832304812a05a289feeb6d22a" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 32865 } } }
I20260808 17:01:03.587524 5308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:03.587651 5306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:03.587949 5320 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:03.588474 5320 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:03.589922 5320 catalog_manager.cc:1347] Generated new cluster ID: a326e7b9431c44f1b3a91ef010be9a74
I20260808 17:01:03.589982 5320 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:03.593321 5316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.587411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49046 (local address 127.8.14.126:32865)
0808 17:01:03.588242 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:03.588251 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:03.588282 (+ 31us) server_negotiation.cc:407] Connection header received
0808 17:01:03.588396 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:03.588402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:03.588469 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:03.588884 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:03.588975 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.590476 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.591276 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.591857 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.591966 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.592608 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:03.592612 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:03.592612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:03.592799 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:03.592843 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:03.592856 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:03.592917 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:03.593029 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 17:01:03.593115 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":159,"threads_started":1}
I20260808 17:01:03.603719 5320 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:03.604383 5320 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:03.607182 5320 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a: Generated new TSK 0
I20260808 17:01:03.607365 5320 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:03.615789 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.65:0
--local_ip_for_outbound_sockets=127.8.14.65
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:32865
--builtin_ntp_servers=127.8.14.84:41993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 17:01:03.703091 5335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:03.703282 5335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:03.703302 5335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:03.704432 5335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:03.704473 5335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:03.704499 5335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.65
I20260808 17:01:03.705847 5335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.14.65
--webserver_port=0
--tserver_master_addrs=127.8.14.126:32865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.65
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:03.706099 5335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:03.706416 5335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:03.712152 5340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:03.712169 5341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:03.712209 5345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:03.712335 5335 server_base.cc:1034] running on GCE node
I20260808 17:01:03.712642 5335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:03.712901 5335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:03.714068 5335 hybrid_clock.cc:648] HybridClock initialized: now 1786208463714052 us; error 34 us; skew 500 ppm
I20260808 17:01:03.715356 5335 webserver.cc:460] Webserver started at http://127.8.14.65:33095/ using document root <none> and password file <none>
I20260808 17:01:03.715608 5335 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:03.715680 5335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:03.715830 5335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:03.716863 5335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-0/data/instance:
uuid: "4bfdca2550584bd785edaf140e4d12e6"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "8b7e6c9664b6d5c4ee8d7c4055514385"
server_key_iv: "6efc1c6a6de62e0c6c51c597e8c767f3"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.717223 5335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance:
uuid: "4bfdca2550584bd785edaf140e4d12e6"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "8b7e6c9664b6d5c4ee8d7c4055514385"
server_key_iv: "6efc1c6a6de62e0c6c51c597e8c767f3"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.718613 5335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:01:03.719485 5353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:03.719657 5335 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 17:01:03.719759 5335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-0/wal
uuid: "4bfdca2550584bd785edaf140e4d12e6"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "8b7e6c9664b6d5c4ee8d7c4055514385"
server_key_iv: "6efc1c6a6de62e0c6c51c597e8c767f3"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.719846 5335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:03.740859 5335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:03.741183 5335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:03.741343 5335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:03.741575 5335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:03.741912 5335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:03.741971 5335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:03.742014 5335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:03.742041 5335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:03.747822 5335 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.65:38833
I20260808 17:01:03.747920 5514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.65:38833 every 8 connection(s)
I20260808 17:01:03.748240 5335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-0/data/info.pb
I20260808 17:01:03.752210 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 5335
I20260808 17:01:03.752317 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-0/wal/instance
I20260808 17:01:03.752560 8249 external_mini_cluster.cc:1428] Setting key a15446bc4e9cffeec4a7566a7f7b69af
I20260808 17:01:03.753630 5316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.749016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:58061 (local address 127.8.14.126:32865)
0808 17:01:03.749089 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:03.749091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:03.749305 (+ 214us) server_negotiation.cc:407] Connection header received
0808 17:01:03.749783 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:03.749784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:03.749796 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:03.749861 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:03.750508 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.751067 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.752012 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.752341 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.752415 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.753347 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:03.753348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:03.753349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:03.753354 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:03.753373 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:03.753378 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:03.753436 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:03.753549 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 17:01:03.753573 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 17:01:03.753717 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.66:0
--local_ip_for_outbound_sockets=127.8.14.66
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:32865
--builtin_ntp_servers=127.8.14.84:41993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 17:01:03.753993 5520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.749018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:32865 (local address 127.8.14.65:58061)
0808 17:01:03.749184 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:03.749217 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:03.749702 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:03.749915 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:03.749918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:03.749926 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:03.750437 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.750441 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.751148 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.751148 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.751908 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.751910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.752441 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.752442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.752554 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.753158 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:01:03.753158 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:03.753215 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:01:03.753245 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:03.753453 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:03.753460 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:03.753463 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:03.753536 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 17:01:03.753647 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260808 17:01:03.754542 5516 heartbeater.cc:347] Connected to a master server at 127.8.14.126:32865
I20260808 17:01:03.754647 5516 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:03.754853 5516 heartbeater.cc:511] Master 127.8.14.126:32865 requested a full tablet report, sending...
I20260808 17:01:03.755404 5258 ts_manager.cc:194] Registered new tserver with Master: 4bfdca2550584bd785edaf140e4d12e6 (127.8.14.65:38833)
I20260808 17:01:03.756019 5258 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.14.65:58061
W20260808 17:01:03.836858 5521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:03.837047 5521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:03.837069 5521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:03.838121 5521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:03.838153 5521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:03.838181 5521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.66
I20260808 17:01:03.839550 5521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.14.66
--webserver_port=0
--tserver_master_addrs=127.8.14.126:32865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.66
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:03.839725 5521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:03.839967 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:03.844851 5531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:03.844867 5533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:03.844882 5530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:03.845170 5521 server_base.cc:1034] running on GCE node
I20260808 17:01:03.845405 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:03.845657 5521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:03.846820 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786208463846801 us; error 30 us; skew 500 ppm
I20260808 17:01:03.848145 5521 webserver.cc:460] Webserver started at http://127.8.14.66:42629/ using document root <none> and password file <none>
I20260808 17:01:03.848408 5521 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:03.848481 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:03.848634 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:03.849656 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/data/instance:
uuid: "297894ae43e14aceaefc9bec58bc1864"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "4bfab7f0b8fcbd6d93b425965f0e9d96"
server_key_iv: "6d1ea42029c441788a7024be5c1f033a"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.850052 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance:
uuid: "297894ae43e14aceaefc9bec58bc1864"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "4bfab7f0b8fcbd6d93b425965f0e9d96"
server_key_iv: "6d1ea42029c441788a7024be5c1f033a"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.851430 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:01:03.852142 5540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:03.852320 5521 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:03.852413 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/wal
uuid: "297894ae43e14aceaefc9bec58bc1864"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "4bfab7f0b8fcbd6d93b425965f0e9d96"
server_key_iv: "6d1ea42029c441788a7024be5c1f033a"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.852490 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:03.867730 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:03.868005 5521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:03.868155 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:03.868387 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:03.868700 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:03.868752 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:03.868793 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:03.868820 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:03.874423 5521 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.66:35699
I20260808 17:01:03.874493 5707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.66:35699 every 8 connection(s)
I20260808 17:01:03.874811 5521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/data/info.pb
I20260808 17:01:03.879724 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 5521
I20260808 17:01:03.879853 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/wal/instance
I20260808 17:01:03.879954 5316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.875647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.66:44657 (local address 127.8.14.126:32865)
0808 17:01:03.875723 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:03.875725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:03.875958 (+ 233us) server_negotiation.cc:407] Connection header received
0808 17:01:03.876416 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:03.876417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:03.876427 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:03.876481 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:03.877125 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.877607 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.878532 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.878710 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.878761 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.879616 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:03.879618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:03.879619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:03.879624 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:03.879641 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:03.879647 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:03.879709 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:03.879793 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 17:01:03.879827 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 17:01:03.880117 8249 external_mini_cluster.cc:1428] Setting key 61d09dda92d69747b99e0fbc7524b7bc
I20260808 17:01:03.880167 5710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:03.875609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:32865 (local address 127.8.14.66:44657)
0808 17:01:03.875827 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:03.875867 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:03.876334 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:03.876539 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:03.876558 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:03.876567 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:03.877051 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.877056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.877689 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.877690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.878455 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:03.878457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:03.878786 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:03.878787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:03.878891 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:03.879453 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:01:03.879453 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:03.879481 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:01:03.879529 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:03.879723 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:03.879727 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:03.879729 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:01:03.879775 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 17:01:03.879909 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":121,"threads_started":1}
I20260808 17:01:03.880652 5708 heartbeater.cc:347] Connected to a master server at 127.8.14.126:32865
I20260808 17:01:03.880756 5708 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:03.880926 5708 heartbeater.cc:511] Master 127.8.14.126:32865 requested a full tablet report, sending...
I20260808 17:01:03.881315 5257 ts_manager.cc:194] Registered new tserver with Master: 297894ae43e14aceaefc9bec58bc1864 (127.8.14.66:35699)
I20260808 17:01:03.881345 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.14.67:0
--local_ip_for_outbound_sockets=127.8.14.67
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:32865
--builtin_ntp_servers=127.8.14.84:41993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 17:01:03.881745 5257 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.14.66:44657
W20260808 17:01:03.963706 5711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:03.963898 5711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:03.963920 5711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:03.965071 5711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:03.965114 5711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:03.965139 5711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.14.67
I20260808 17:01:03.966471 5711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.14.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.14.67
--webserver_port=0
--tserver_master_addrs=127.8.14.126:32865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.14.67
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:03.966722 5711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:03.967069 5711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:03.971853 5719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:03.971853 5724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:03.971910 5720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:03.972043 5711 server_base.cc:1034] running on GCE node
I20260808 17:01:03.972306 5711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:03.972565 5711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:03.973726 5711 hybrid_clock.cc:648] HybridClock initialized: now 1786208463973719 us; error 35 us; skew 500 ppm
I20260808 17:01:03.975102 5711 webserver.cc:460] Webserver started at http://127.8.14.67:42741/ using document root <none> and password file <none>
I20260808 17:01:03.975356 5711 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:03.975430 5711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:03.975579 5711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:03.976591 5711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/data/instance:
uuid: "67e0b5a3ea574a39a63fc2c5dce186d7"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "e1a1dc74e9bdcb7e2e137a96904d2f00"
server_key_iv: "14b1f1b5ffad4152eaa3b398de701671"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.976955 5711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance:
uuid: "67e0b5a3ea574a39a63fc2c5dce186d7"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "e1a1dc74e9bdcb7e2e137a96904d2f00"
server_key_iv: "14b1f1b5ffad4152eaa3b398de701671"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.978238 5711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:01:03.978989 5732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:03.979161 5711 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:03.979259 5711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/wal
uuid: "67e0b5a3ea574a39a63fc2c5dce186d7"
format_stamp: "Formatted at 2026-08-08 17:01:03 on dist-test-slave-1nl6"
server_key: "e1a1dc74e9bdcb7e2e137a96904d2f00"
server_key_iv: "14b1f1b5ffad4152eaa3b398de701671"
server_key_version: "encryptionkey@0"
I20260808 17:01:03.979346 5711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:03.998503 5711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:03.998874 5711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:03.999060 5711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:03.999300 5711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 17:01:03.999639 5711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 17:01:03.999698 5711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:03.999742 5711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 17:01:03.999776 5711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:04.005432 5711 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.67:41739
I20260808 17:01:04.005511 5899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.67:41739 every 8 connection(s)
I20260808 17:01:04.005851 5711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/data/info.pb
I20260808 17:01:04.007207 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 5711
I20260808 17:01:04.007321 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/wal/instance
I20260808 17:01:04.007546 8249 external_mini_cluster.cc:1428] Setting key cb8bf65ec397e154043950bcba67052a
I20260808 17:01:04.011442 5316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:04.006630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.67:51141 (local address 127.8.14.126:32865)
0808 17:01:04.006676 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:04.006678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:04.006953 (+ 275us) server_negotiation.cc:407] Connection header received
0808 17:01:04.007602 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:04.007603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:04.007617 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:04.007670 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:04.008499 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.009011 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.009981 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.010187 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.010265 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:04.011150 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:04.011152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:04.011152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:04.011157 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:04.011173 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:04.011177 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:04.011246 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:04.011335 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 17:01:04.011364 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:01:04.011653 5902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:04.006610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:32865 (local address 127.8.14.67:51141)
0808 17:01:04.006811 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:04.006848 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:04.007489 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:04.007741 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:04.007745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:04.007754 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:04.008409 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:04.008414 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.009112 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.009113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:04.009890 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:04.009892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.010274 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.010275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:04.010418 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:04.010995 (+ 577us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 17:01:04.010995 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:04.011023 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 17:01:04.011046 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:04.011260 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:04.011265 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:04.011268 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:04.011328 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 17:01:04.011433 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
I20260808 17:01:04.012140 5900 heartbeater.cc:347] Connected to a master server at 127.8.14.126:32865
I20260808 17:01:04.012235 5900 heartbeater.cc:464] Registering TS with master...
I20260808 17:01:04.012377 5900 heartbeater.cc:511] Master 127.8.14.126:32865 requested a full tablet report, sending...
I20260808 17:01:04.012725 5257 ts_manager.cc:194] Registered new tserver with Master: 67e0b5a3ea574a39a63fc2c5dce186d7 (127.8.14.67:41739)
I20260808 17:01:04.013055 5257 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.14.67:51141
I20260808 17:01:04.019201 8249 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 17:01:04.023270 5316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:04.020565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49054 (local address 127.8.14.126:32865)
0808 17:01:04.020634 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:04.020636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:04.020713 (+ 77us) server_negotiation.cc:407] Connection header received
0808 17:01:04.020753 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:04.020754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:04.020763 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:04.020812 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:04.020959 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.021404 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.022118 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.022295 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.022350 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:04.023055 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:04.023056 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:04.023056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:04.023060 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:04.023072 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:04.023076 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:04.023145 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:04.023185 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 17:01:04.023208 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 17:01:04.024783 5257 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 17:01:04.026333 5257 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 17:01:04.032711 5915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:04.028594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:41739 (local address 127.0.0.1:45406)
0808 17:01:04.028754 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:04.028768 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:04.028827 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:04.029300 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:04.029302 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:04.029304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:04.029392 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:04.029395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.030372 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.030373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:04.031613 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:04.031615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.032410 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.032411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:04.032489 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:04.032503 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 17:01:04.032569 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 17:01:04.032620 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":2,"thread_start_us":106,"threads_started":1}
I20260808 17:01:04.032817 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:04.028768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37854 (local address 127.8.14.65:38833)
0808 17:01:04.029089 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:04.029095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:04.029106 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:01:04.029149 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:04.029151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:04.029168 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:04.029285 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:04.029456 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.030558 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.031777 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.032497 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.032577 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:04.032662 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:04.032690 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 17:01:04.032727 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":93,"threads_started":1}
I20260808 17:01:04.032825 5913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:04.028590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:38833 (local address 127.0.0.1:37854)
0808 17:01:04.028723 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:04.028765 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:04.028839 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:04.029300 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:04.029303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:04.029306 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:04.029377 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:04.029379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.030650 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.030650 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:04.031706 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:04.031708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.032575 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.032575 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:04.032627 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:04.032630 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:04.032672 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 17:01:04.032702 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260808 17:01:04.033183 5919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:04.028958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45406 (local address 127.8.14.67:41739)
0808 17:01:04.029119 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:04.029124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:04.029134 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:01:04.029173 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:04.029175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:04.029191 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:04.029475 (+ 284us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:04.029530 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.030270 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.031702 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.032312 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.032397 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:04.032487 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:04.033048 (+ 561us) server_negotiation.cc:299] Negotiation successful
0808 17:01:04.033097 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260808 17:01:04.033285 5917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:04.028818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:35699 (local address 127.0.0.1:57488)
0808 17:01:04.029306 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:04.029316 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:04.029362 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:04.029634 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:04.029635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:04.029637 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:04.029671 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:04.029672 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.030884 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.030885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:04.031881 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:04.031884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.033037 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.033038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:04.033116 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:04.033122 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 17:01:04.033174 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 17:01:04.033210 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":84,"threads_started":1}
I20260808 17:01:04.033666 5920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:04.029042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57488 (local address 127.8.14.66:35699)
0808 17:01:04.029218 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:04.029222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:04.029428 (+ 206us) server_negotiation.cc:407] Connection header received
0808 17:01:04.029477 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:04.029479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:04.029494 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:04.029566 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:04.029766 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.030790 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.031978 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.032716 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.032781 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:04.033509 (+ 728us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:04.033537 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 17:01:04.033577 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":76,"threads_started":1}
I20260808 17:01:04.033679 5422 tablet_service.cc:1467] Processing CreateTablet for tablet 40796173b45f4e66bce1e7d5ce5d408b (DEFAULT_TABLE table=test-table [id=5069e548cd624f87acf603254215dc46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:04.033902 5803 tablet_service.cc:1467] Processing CreateTablet for tablet 40796173b45f4e66bce1e7d5ce5d408b (DEFAULT_TABLE table=test-table [id=5069e548cd624f87acf603254215dc46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:04.034076 5422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40796173b45f4e66bce1e7d5ce5d408b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:04.034245 5803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40796173b45f4e66bce1e7d5ce5d408b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:04.034284 5606 tablet_service.cc:1467] Processing CreateTablet for tablet 40796173b45f4e66bce1e7d5ce5d408b (DEFAULT_TABLE table=test-table [id=5069e548cd624f87acf603254215dc46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 17:01:04.034605 5606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40796173b45f4e66bce1e7d5ce5d408b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:04.036733 5921 tablet_bootstrap.cc:492] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6: Bootstrap starting.
I20260808 17:01:04.037395 5921 tablet_bootstrap.cc:654] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:04.037616 5922 tablet_bootstrap.cc:492] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7: Bootstrap starting.
I20260808 17:01:04.037724 5921 log.cc:824] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:04.038204 5925 tablet_bootstrap.cc:492] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864: Bootstrap starting.
I20260808 17:01:04.038264 5922 tablet_bootstrap.cc:654] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:04.038576 5922 log.cc:824] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:04.038866 5925 tablet_bootstrap.cc:654] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:04.039239 5925 log.cc:824] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:04.039425 5922 tablet_bootstrap.cc:492] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7: No bootstrap required, opened a new log
I20260808 17:01:04.039506 5921 tablet_bootstrap.cc:492] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6: No bootstrap required, opened a new log
I20260808 17:01:04.039517 5922 ts_tablet_manager.cc:1397] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:01:04.039573 5921 ts_tablet_manager.cc:1397] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 17:01:04.039976 5925 tablet_bootstrap.cc:492] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864: No bootstrap required, opened a new log
I20260808 17:01:04.040059 5925 ts_tablet_manager.cc:1397] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 17:01:04.042045 5925 raft_consensus.cc:348] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfdca2550584bd785edaf140e4d12e6" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38833 } } peers { permanent_uuid: "67e0b5a3ea574a39a63fc2c5dce186d7" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 41739 } } peers { permanent_uuid: "297894ae43e14aceaefc9bec58bc1864" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35699 } }
I20260808 17:01:04.042207 5925 raft_consensus.cc:374] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:04.042253 5925 raft_consensus.cc:729] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 297894ae43e14aceaefc9bec58bc1864, State: Initialized, Role: FOLLOWER
I20260808 17:01:04.042157 5922 raft_consensus.cc:348] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfdca2550584bd785edaf140e4d12e6" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38833 } } peers { permanent_uuid: "67e0b5a3ea574a39a63fc2c5dce186d7" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 41739 } } peers { permanent_uuid: "297894ae43e14aceaefc9bec58bc1864" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35699 } }
I20260808 17:01:04.042157 5921 raft_consensus.cc:348] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfdca2550584bd785edaf140e4d12e6" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38833 } } peers { permanent_uuid: "67e0b5a3ea574a39a63fc2c5dce186d7" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 41739 } } peers { permanent_uuid: "297894ae43e14aceaefc9bec58bc1864" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35699 } }
I20260808 17:01:04.042320 5922 raft_consensus.cc:374] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:04.042320 5921 raft_consensus.cc:374] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:04.042369 5922 raft_consensus.cc:729] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67e0b5a3ea574a39a63fc2c5dce186d7, State: Initialized, Role: FOLLOWER
I20260808 17:01:04.042404 5921 raft_consensus.cc:729] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bfdca2550584bd785edaf140e4d12e6, State: Initialized, Role: FOLLOWER
I20260808 17:01:04.042369 5925 consensus_queue.cc:260] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfdca2550584bd785edaf140e4d12e6" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38833 } } peers { permanent_uuid: "67e0b5a3ea574a39a63fc2c5dce186d7" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 41739 } } peers { permanent_uuid: "297894ae43e14aceaefc9bec58bc1864" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35699 } }
I20260808 17:01:04.042498 5922 consensus_queue.cc:260] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfdca2550584bd785edaf140e4d12e6" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38833 } } peers { permanent_uuid: "67e0b5a3ea574a39a63fc2c5dce186d7" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 41739 } } peers { permanent_uuid: "297894ae43e14aceaefc9bec58bc1864" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35699 } }
I20260808 17:01:04.042520 5921 consensus_queue.cc:260] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfdca2550584bd785edaf140e4d12e6" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38833 } } peers { permanent_uuid: "67e0b5a3ea574a39a63fc2c5dce186d7" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 41739 } } peers { permanent_uuid: "297894ae43e14aceaefc9bec58bc1864" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35699 } }
I20260808 17:01:04.042631 5925 ts_tablet_manager.cc:1428] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:01:04.042739 5708 heartbeater.cc:503] Master 127.8.14.126:32865 was elected leader, sending a full tablet report...
I20260808 17:01:04.042752 5922 ts_tablet_manager.cc:1428] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 17:01:04.042752 5921 ts_tablet_manager.cc:1428] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 17:01:04.042866 5900 heartbeater.cc:503] Master 127.8.14.126:32865 was elected leader, sending a full tablet report...
I20260808 17:01:04.042927 5516 heartbeater.cc:503] Master 127.8.14.126:32865 was elected leader, sending a full tablet report...
W20260808 17:01:04.125396 5709 tablet.cc:2369] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 17:01:04.162701 5932 raft_consensus.cc:482] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 17:01:04.162830 5932 raft_consensus.cc:504] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfdca2550584bd785edaf140e4d12e6" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38833 } } peers { permanent_uuid: "67e0b5a3ea574a39a63fc2c5dce186d7" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 41739 } } peers { permanent_uuid: "297894ae43e14aceaefc9bec58bc1864" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35699 } }
I20260808 17:01:04.163259 5932 leader_election.cc:288] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 67e0b5a3ea574a39a63fc2c5dce186d7 (127.8.14.67:41739), 297894ae43e14aceaefc9bec58bc1864 (127.8.14.66:35699)
I20260808 17:01:04.165949 5919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:04.163550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:41937 (local address 127.8.14.67:41739)
0808 17:01:04.163595 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:04.163596 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:04.163605 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:01:04.163645 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:04.163646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:04.163654 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:04.163721 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:04.163890 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.164372 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.165278 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.165620 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.165699 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:04.165710 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:04.165853 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 17:01:04.165889 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:01:04.165949 5520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:04.163446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.67:41739 (local address 127.8.14.65:41937)
0808 17:01:04.163508 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:04.163519 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:04.163564 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:04.163726 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:04.163727 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:04.163728 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:04.163813 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:04.163815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.164469 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.164470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:04.165176 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:04.165178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.165699 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.165700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:04.165770 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:04.165773 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 17:01:04.165834 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 17:01:04.165877 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 17:01:04.166272 5933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:04.163455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.66:35699 (local address 127.8.14.65:48805)
0808 17:01:04.163653 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:04.163661 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:04.163743 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:04.163918 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:04.163919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:04.163920 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 17:01:04.163996 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:04.163998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.164626 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.164627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:04.165339 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:04.165341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.165892 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.165893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:04.165955 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:04.165957 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 17:01:04.166155 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 17:01:04.166197 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":2,"thread_start_us":121,"threads_started":1}
I20260808 17:01:04.166337 5920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:04.163533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.14.65:48805 (local address 127.8.14.66:35699)
0808 17:01:04.163584 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:04.163587 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:04.163727 (+ 140us) server_negotiation.cc:407] Connection header received
0808 17:01:04.163828 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:04.163829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:04.163850 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:04.163913 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 17:01:04.164069 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.164542 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.165438 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.165773 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.165869 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:04.165878 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:04.166207 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 17:01:04.166258 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:01:04.166306 5832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40796173b45f4e66bce1e7d5ce5d408b" candidate_uuid: "4bfdca2550584bd785edaf140e4d12e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67e0b5a3ea574a39a63fc2c5dce186d7" is_pre_election: true
I20260808 17:01:04.166458 5832 raft_consensus.cc:2454] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bfdca2550584bd785edaf140e4d12e6 in term 0.
I20260808 17:01:04.166757 5356 leader_election.cc:302] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bfdca2550584bd785edaf140e4d12e6, 67e0b5a3ea574a39a63fc2c5dce186d7; no voters:
I20260808 17:01:04.166741 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40796173b45f4e66bce1e7d5ce5d408b" candidate_uuid: "4bfdca2550584bd785edaf140e4d12e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297894ae43e14aceaefc9bec58bc1864" is_pre_election: true
I20260808 17:01:04.166890 5630 raft_consensus.cc:2454] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bfdca2550584bd785edaf140e4d12e6 in term 0.
I20260808 17:01:04.166942 5932 raft_consensus.cc:2781] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 17:01:04.166999 5932 raft_consensus.cc:482] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 17:01:04.167034 5932 raft_consensus.cc:3037] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:04.167758 5932 raft_consensus.cc:504] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfdca2550584bd785edaf140e4d12e6" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38833 } } peers { permanent_uuid: "67e0b5a3ea574a39a63fc2c5dce186d7" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 41739 } } peers { permanent_uuid: "297894ae43e14aceaefc9bec58bc1864" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35699 } }
I20260808 17:01:04.167924 5932 leader_election.cc:288] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [CANDIDATE]: Term 1 election: Requested vote from peers 67e0b5a3ea574a39a63fc2c5dce186d7 (127.8.14.67:41739), 297894ae43e14aceaefc9bec58bc1864 (127.8.14.66:35699)
I20260808 17:01:04.168069 5832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40796173b45f4e66bce1e7d5ce5d408b" candidate_uuid: "4bfdca2550584bd785edaf140e4d12e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67e0b5a3ea574a39a63fc2c5dce186d7"
I20260808 17:01:04.168076 5630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40796173b45f4e66bce1e7d5ce5d408b" candidate_uuid: "4bfdca2550584bd785edaf140e4d12e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "297894ae43e14aceaefc9bec58bc1864"
I20260808 17:01:04.168151 5832 raft_consensus.cc:3037] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:04.168151 5630 raft_consensus.cc:3037] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:04.169062 5630 raft_consensus.cc:2454] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bfdca2550584bd785edaf140e4d12e6 in term 1.
I20260808 17:01:04.169062 5832 raft_consensus.cc:2454] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bfdca2550584bd785edaf140e4d12e6 in term 1.
I20260808 17:01:04.169239 5356 leader_election.cc:302] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bfdca2550584bd785edaf140e4d12e6, 67e0b5a3ea574a39a63fc2c5dce186d7; no voters:
I20260808 17:01:04.169353 5932 raft_consensus.cc:2781] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:04.169499 5932 raft_consensus.cc:686] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [term 1 LEADER]: Becoming Leader. State: Replica: 4bfdca2550584bd785edaf140e4d12e6, State: Running, Role: LEADER
I20260808 17:01:04.169596 5932 consensus_queue.cc:237] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfdca2550584bd785edaf140e4d12e6" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38833 } } peers { permanent_uuid: "67e0b5a3ea574a39a63fc2c5dce186d7" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 41739 } } peers { permanent_uuid: "297894ae43e14aceaefc9bec58bc1864" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35699 } }
I20260808 17:01:04.170457 5257 catalog_manager.cc:5495] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bfdca2550584bd785edaf140e4d12e6 (127.8.14.65). New cstate: current_term: 1 leader_uuid: "4bfdca2550584bd785edaf140e4d12e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bfdca2550584bd785edaf140e4d12e6" member_type: VOTER last_known_addr { host: "127.8.14.65" port: 38833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67e0b5a3ea574a39a63fc2c5dce186d7" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 41739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "297894ae43e14aceaefc9bec58bc1864" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35699 } health_report { overall_health: UNKNOWN } } }
I20260808 17:01:04.177791 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:04.175410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37860 (local address 127.8.14.65:38833)
0808 17:01:04.175456 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:04.175458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:04.175516 (+ 58us) server_negotiation.cc:407] Connection header received
0808 17:01:04.175588 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:04.175589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:04.175617 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:04.175694 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:04.175831 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.176259 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.177005 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:04.177190 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:04.177267 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:04.177451 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:04.177543 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:04.177629 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:04.177673 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 17:01:04.177692 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:01:04.179562 5630 raft_consensus.cc:1261] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864 [term 1 FOLLOWER]: Refusing update from remote peer 4bfdca2550584bd785edaf140e4d12e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:04.179562 5832 raft_consensus.cc:1261] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7 [term 1 FOLLOWER]: Refusing update from remote peer 4bfdca2550584bd785edaf140e4d12e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:04.179910 5932 consensus_queue.cc:1035] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "297894ae43e14aceaefc9bec58bc1864" member_type: VOTER last_known_addr { host: "127.8.14.66" port: 35699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:04.180042 5934 consensus_queue.cc:1035] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67e0b5a3ea574a39a63fc2c5dce186d7" member_type: VOTER last_known_addr { host: "127.8.14.67" port: 41739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:04.182742 5940 mvcc.cc:205] Tried to move back new op lower bound from 7316309869276422144 to 7316309869239267328. Current Snapshot: MvccSnapshot[applied={T|T < 7316309869276422144}]
I20260808 17:01:04.184629 5258 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:04.184705 5258 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 17:01:04.185397 5258 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7deed45832304812a05a289feeb6d22a: Sending DeleteTablet for 3 replicas of tablet 40796173b45f4e66bce1e7d5ce5d408b
I20260808 17:01:04.185748 5422 tablet_service.cc:1514] Processing DeleteTablet for tablet 40796173b45f4e66bce1e7d5ce5d408b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:04 UTC) from {username='slave'} at 127.0.0.1:37854
I20260808 17:01:04.185820 5606 tablet_service.cc:1514] Processing DeleteTablet for tablet 40796173b45f4e66bce1e7d5ce5d408b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:04 UTC) from {username='slave'} at 127.0.0.1:57488
I20260808 17:01:04.185874 5803 tablet_service.cc:1514] Processing DeleteTablet for tablet 40796173b45f4e66bce1e7d5ce5d408b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 17:01:04 UTC) from {username='slave'} at 127.0.0.1:45406
I20260808 17:01:04.186041 5951 tablet_replica.cc:330] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6: stopping tablet replica
I20260808 17:01:04.186144 5951 raft_consensus.cc:2229] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 17:01:04.186236 5951 raft_consensus.cc:2258] T 40796173b45f4e66bce1e7d5ce5d408b P 4bfdca2550584bd785edaf140e4d12e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:04.186403 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 5335
I20260808 17:01:04.191644 5954 tablet_replica.cc:330] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864: stopping tablet replica
I20260808 17:01:04.191754 5954 raft_consensus.cc:2229] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:04.191859 5954 raft_consensus.cc:2258] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:04.192178 5953 tablet_replica.cc:330] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7: stopping tablet replica
I20260808 17:01:04.192288 5953 raft_consensus.cc:2229] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 17:01:04.192314 5954 ts_tablet_manager.cc:1905] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:04.192435 5953 raft_consensus.cc:2258] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 17:01:04.193394 5953 ts_tablet_manager.cc:1905] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 17:01:04.194046 5954 ts_tablet_manager.cc:1918] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:01:04.194139 5954 log.cc:1196] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-1/wal/wals/40796173b45f4e66bce1e7d5ce5d408b
I20260808 17:01:04.194391 5954 ts_tablet_manager.cc:1939] T 40796173b45f4e66bce1e7d5ce5d408b P 297894ae43e14aceaefc9bec58bc1864: Deleting consensus metadata
I20260808 17:01:04.194826 5242 catalog_manager.cc:4841] TS 297894ae43e14aceaefc9bec58bc1864 (127.8.14.66:35699): tablet 40796173b45f4e66bce1e7d5ce5d408b (table test-table [id=5069e548cd624f87acf603254215dc46]) successfully deleted
I20260808 17:01:04.195477 5953 ts_tablet_manager.cc:1918] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 17:01:04.195555 5953 log.cc:1196] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7: Deleting WAL directory at /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786208190438830-8249-0/minicluster-data/ts-2/wal/wals/40796173b45f4e66bce1e7d5ce5d408b
I20260808 17:01:04.195801 5953 ts_tablet_manager.cc:1939] T 40796173b45f4e66bce1e7d5ce5d408b P 67e0b5a3ea574a39a63fc2c5dce186d7: Deleting consensus metadata
I20260808 17:01:04.196193 5239 catalog_manager.cc:4841] TS 67e0b5a3ea574a39a63fc2c5dce186d7 (127.8.14.67:41739): tablet 40796173b45f4e66bce1e7d5ce5d408b (table test-table [id=5069e548cd624f87acf603254215dc46]) successfully deleted
W20260808 17:01:04.199973 5242 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.14.65:38833 (error 108)
I20260808 17:01:04.200233 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 5521
I20260808 17:01:04.203477 5917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:04.200334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.65:38833 (local address 127.0.0.1:37862)
0808 17:01:04.203381 (+ 3047us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:04.203435 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.65:38833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3021}
W20260808 17:01:04.203641 5242 catalog_manager.cc:4584] TS 4bfdca2550584bd785edaf140e4d12e6 (127.8.14.65:38833): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 40796173b45f4e66bce1e7d5ce5d408b: Network error: Client connection negotiation failed: client connection to 127.8.14.65:38833: connect: Connection refused (error 111)
I20260808 17:01:04.208458 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 5711
I20260808 17:01:04.218437 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 5217
2026-08-08T17:01:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (786 ms)
[----------] 6 tests from EncryptionPolicyTest (9163 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:04 dist-test-slave-1nl6 krb5kdc[5959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:04 dist-test-slave-1nl6 krb5kdc[5959](info): set up 2 sockets
Aug 08 17:01:04 dist-test-slave-1nl6 krb5kdc[5959](info): commencing operation
krb5kdc: starting...
W20260808 17:01:05.439862 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:05 dist-test-slave-1nl6 krb5kdc[5959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:01:05.559393 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45355
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:05.641511 5979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:05.641695 5979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:05.641716 5979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:05.642900 5979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:05.642964 5979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:05.643018 5979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:05.643036 5979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:05.643061 5979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:05.644595 5979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45355
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:05.644809 5979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:05.645056 5979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:05.649919 5992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:05.649919 5987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:05.650000 5986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:05.650285 5979 server_base.cc:1034] running on GCE node
I20260808 17:01:05.650542 5979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:05.650817 5979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:05.651984 5979 hybrid_clock.cc:648] HybridClock initialized: now 1786208465651989 us; error 50 us; skew 500 ppm
Aug 08 17:01:05 dist-test-slave-1nl6 krb5kdc[5959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208465, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:05.660777 5979 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:01:05.661273 5979 webserver.cc:460] Webserver started at http://127.0.0.1:35547/ using document root <none> and password file <none>
I20260808 17:01:05.661535 5979 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:05.661605 5979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:05.661762 5979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:05.662793 5979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "97371f821a8849cc905431fe1f76d865"
format_stamp: "Formatted at 2026-08-08 17:01:05 on dist-test-slave-1nl6"
server_key: "9cdf22d3a0c379f80d5cac21d5c4dfec"
server_key_iv: "0fc1c6164e537bbd5a200e6d801e3390"
server_key_version: "encryptionkey@0"
I20260808 17:01:05.663275 5979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "97371f821a8849cc905431fe1f76d865"
format_stamp: "Formatted at 2026-08-08 17:01:05 on dist-test-slave-1nl6"
server_key: "9cdf22d3a0c379f80d5cac21d5c4dfec"
server_key_iv: "0fc1c6164e537bbd5a200e6d801e3390"
server_key_version: "encryptionkey@0"
I20260808 17:01:05.664671 5979 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:05.665392 6001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:05.665588 5979 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:05.665681 5979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "97371f821a8849cc905431fe1f76d865"
format_stamp: "Formatted at 2026-08-08 17:01:05 on dist-test-slave-1nl6"
server_key: "9cdf22d3a0c379f80d5cac21d5c4dfec"
server_key_iv: "0fc1c6164e537bbd5a200e6d801e3390"
server_key_version: "encryptionkey@0"
I20260808 17:01:05.665737 5979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:05.685212 5979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:05.687413 5979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:05.687522 5979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:05.691318 5979 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45355
I20260808 17:01:05.691390 6074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45355 every 8 connection(s)
I20260808 17:01:05.691707 5979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:05.692721 6075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:05.694437 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 5979
I20260808 17:01:05.694528 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:05.694728 8249 external_mini_cluster.cc:1428] Setting key b6f508f98ae953d22776860bffeef5c6
I20260808 17:01:05.695163 6075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865: Bootstrap starting.
I20260808 17:01:05.695926 6075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:05.696272 6075 log.cc:824] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:05.697047 6075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865: No bootstrap required, opened a new log
I20260808 17:01:05.699180 6075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97371f821a8849cc905431fe1f76d865" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45355 } }
I20260808 17:01:05.699312 6075 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:05.699335 6075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97371f821a8849cc905431fe1f76d865, State: Initialized, Role: FOLLOWER
I20260808 17:01:05.699468 6075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97371f821a8849cc905431fe1f76d865" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45355 } }
I20260808 17:01:05.699561 6075 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:05.699605 6075 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:05.699689 6075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:05.700416 6075 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97371f821a8849cc905431fe1f76d865" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45355 } }
I20260808 17:01:05.700549 6075 leader_election.cc:302] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97371f821a8849cc905431fe1f76d865; no voters:
I20260808 17:01:05.700735 6075 leader_election.cc:288] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:05.700796 6084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:05.700947 6084 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [term 1 LEADER]: Becoming Leader. State: Replica: 97371f821a8849cc905431fe1f76d865, State: Running, Role: LEADER
I20260808 17:01:05.701004 6075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:05.701089 6084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97371f821a8849cc905431fe1f76d865" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45355 } }
I20260808 17:01:05.701496 6084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97371f821a8849cc905431fe1f76d865. Latest consensus state: current_term: 1 leader_uuid: "97371f821a8849cc905431fe1f76d865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97371f821a8849cc905431fe1f76d865" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45355 } } }
I20260808 17:01:05.701740 6084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [sys.catalog]: This master's current role is: LEADER
Aug 08 17:01:05 dist-test-slave-1nl6 krb5kdc[5959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 17:01:05.701664 6085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97371f821a8849cc905431fe1f76d865" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97371f821a8849cc905431fe1f76d865" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45355 } } }
I20260808 17:01:05.702138 6085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:05.702325 6098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:05.703117 6098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:05.705467 6098 catalog_manager.cc:1347] Generated new cluster ID: a07dd741920941579eb164c4949678e1
I20260808 17:01:05.705520 6098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:05.706269 6082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:05.695261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57947 (local address 127.0.0.1:45355)
0808 17:01:05.695446 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:05.695454 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:05.695477 (+ 23us) server_negotiation.cc:407] Connection header received
0808 17:01:05.695991 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:05.696000 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:05.696063 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:05.696173 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:05.696958 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:05.697818 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:05.698988 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:05.699422 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:05.699541 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:05.702451 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:05.702455 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:05.702455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:05.702465 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:05.704397 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:05.704808 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:05.704811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:05.704811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:05.704930 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:05.705200 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:05.705201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:05.705202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:05.705515 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:05.705678 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:05.705973 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 17:01:05.706070 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260808 17:01:05.707284 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:05.713814 6098 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:05.714504 6098 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:05.718273 6082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:05.711154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51545 (local address 127.0.0.1:45355)
0808 17:01:05.711198 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:05.711200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:05.711301 (+ 101us) server_negotiation.cc:407] Connection header received
0808 17:01:05.711329 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:05.711330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:05.711339 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:05.711408 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:05.711966 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:05.712724 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:05.713680 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:05.713999 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:05.714074 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:05.715470 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:05.715471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:05.715472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:05.715479 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:05.717055 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:05.717474 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:05.717475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:05.717475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:05.717537 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:05.717762 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:05.717763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:05.717763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:05.717899 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:05.718006 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:05.718119 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 17:01:05.718182 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:01:05.722803 6098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97371f821a8849cc905431fe1f76d865: Generated new TSK 0
I20260808 17:01:05.723013 6098 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:05.725380 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 5979
W20260808 17:01:05.738487 5957 connection.cc:535] client connection to 127.0.0.1:45355 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45355): Connection reset by peer (error 104)
2026-08-08T17:01:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1511 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:05 dist-test-slave-1nl6 krb5kdc[6119](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:05 dist-test-slave-1nl6 krb5kdc[6119](info): set up 2 sockets
Aug 08 17:01:05 dist-test-slave-1nl6 krb5kdc[6119](info): commencing operation
krb5kdc: starting...
W20260808 17:01:06.941294 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:07 dist-test-slave-1nl6 krb5kdc[6119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:01:07.059479 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43899
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:07.140246 6138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:07.140479 6138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:07.140517 6138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:07.141637 6138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:07.141707 6138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:07.141729 6138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:07.141752 6138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:07.141781 6138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:07.143260 6138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43899
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:07.143543 6138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:07.143769 6138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:07.148551 6148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:07.148535 6150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:07.148536 6147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:07.148561 6138 server_base.cc:1034] running on GCE node
I20260808 17:01:07.149049 6138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:07.149345 6138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:07.150491 6138 hybrid_clock.cc:648] HybridClock initialized: now 1786208467150481 us; error 32 us; skew 500 ppm
Aug 08 17:01:07 dist-test-slave-1nl6 krb5kdc[6119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208467, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:07.152959 6138 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:01:07.153409 6138 webserver.cc:460] Webserver started at http://127.0.0.1:38483/ using document root <none> and password file <none>
I20260808 17:01:07.153638 6138 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:07.153709 6138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:07.153868 6138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:07.154889 6138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "51b2e104c7634fcfb99ef1084afb802a"
format_stamp: "Formatted at 2026-08-08 17:01:07 on dist-test-slave-1nl6"
server_key: "334ab2d65714ab133c2ae688b88e5600"
server_key_iv: "c21eb5d0780abbac00519a5206c06cc3"
server_key_version: "encryptionkey@0"
I20260808 17:01:07.155345 6138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "51b2e104c7634fcfb99ef1084afb802a"
format_stamp: "Formatted at 2026-08-08 17:01:07 on dist-test-slave-1nl6"
server_key: "334ab2d65714ab133c2ae688b88e5600"
server_key_iv: "c21eb5d0780abbac00519a5206c06cc3"
server_key_version: "encryptionkey@0"
I20260808 17:01:07.156705 6138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:01:07.157451 6163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:07.157625 6138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:07.157737 6138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "51b2e104c7634fcfb99ef1084afb802a"
format_stamp: "Formatted at 2026-08-08 17:01:07 on dist-test-slave-1nl6"
server_key: "334ab2d65714ab133c2ae688b88e5600"
server_key_iv: "c21eb5d0780abbac00519a5206c06cc3"
server_key_version: "encryptionkey@0"
I20260808 17:01:07.157819 6138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:07.172685 6138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:07.174563 6138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:07.174723 6138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:07.178872 6138 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43899
I20260808 17:01:07.178954 6237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43899 every 8 connection(s)
I20260808 17:01:07.179312 6138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:07.180330 6239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:07.182569 6239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a: Bootstrap starting.
I20260808 17:01:07.183220 6239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:07.183512 6239 log.cc:824] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:07.183995 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 6138
I20260808 17:01:07.184120 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:07.184288 6239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a: No bootstrap required, opened a new log
I20260808 17:01:07.184357 8249 external_mini_cluster.cc:1428] Setting key 196098fc7d3e81391600cca292a47c2a
I20260808 17:01:07.186936 6239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b2e104c7634fcfb99ef1084afb802a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43899 } }
I20260808 17:01:07.187119 6239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:07.187157 6239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51b2e104c7634fcfb99ef1084afb802a, State: Initialized, Role: FOLLOWER
I20260808 17:01:07.187281 6239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b2e104c7634fcfb99ef1084afb802a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43899 } }
I20260808 17:01:07.187364 6239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:07.187395 6239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:07.187440 6239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:07.188115 6239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b2e104c7634fcfb99ef1084afb802a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43899 } }
I20260808 17:01:07.188205 6239 leader_election.cc:302] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51b2e104c7634fcfb99ef1084afb802a; no voters:
I20260808 17:01:07.188438 6239 leader_election.cc:288] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:07.188458 6247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:07.188580 6247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [term 1 LEADER]: Becoming Leader. State: Replica: 51b2e104c7634fcfb99ef1084afb802a, State: Running, Role: LEADER
I20260808 17:01:07.188709 6239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:07.188717 6247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b2e104c7634fcfb99ef1084afb802a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43899 } }
I20260808 17:01:07.189250 6247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51b2e104c7634fcfb99ef1084afb802a. Latest consensus state: current_term: 1 leader_uuid: "51b2e104c7634fcfb99ef1084afb802a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b2e104c7634fcfb99ef1084afb802a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43899 } } }
I20260808 17:01:07.189339 6247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:07.189256 6248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51b2e104c7634fcfb99ef1084afb802a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51b2e104c7634fcfb99ef1084afb802a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43899 } } }
I20260808 17:01:07.189488 6248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:07.189667 6252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:07.190438 6252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:07.190486 6246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:07.184865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55921 (local address 127.0.0.1:43899)
0808 17:01:07.185054 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:07.185062 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:07.185091 (+ 29us) server_negotiation.cc:407] Connection header received
0808 17:01:07.185630 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:07.185636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:07.185679 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:07.185786 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:07.186014 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:07.186841 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:07.187950 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:07.188463 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:07.188632 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:07.189693 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:07.189697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:07.189698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:07.189920 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:07.189961 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:07.189976 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:07.190058 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:07.190227 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 17:01:07.190262 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260808 17:01:07.191474 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:07.193158 6252 catalog_manager.cc:1347] Generated new cluster ID: 54cf946c29bd4091a097f2a7e596be7c
I20260808 17:01:07.193243 6252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:07.195940 6246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:07.192988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45697 (local address 127.0.0.1:43899)
0808 17:01:07.193022 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:07.193024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:07.193109 (+ 85us) server_negotiation.cc:407] Connection header received
0808 17:01:07.193214 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:07.193215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:07.193225 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:07.193273 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:07.193486 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:07.193935 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:07.194666 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:07.194943 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:07.195015 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:07.195684 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:07.195686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:07.195686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:07.195690 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:07.195704 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:07.195708 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:07.195763 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:07.195837 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 17:01:07.195846 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:01:07.208817 6252 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:07.209319 6252 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:07.215332 6252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51b2e104c7634fcfb99ef1084afb802a: Generated new TSK 0
I20260808 17:01:07.215503 6252 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:07.218983 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 6138
W20260808 17:01:07.229872 6117 connection.cc:535] client connection to 127.0.0.1:43899 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43899): Connection reset by peer (error 104)
2026-08-08T17:01:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:07 dist-test-slave-1nl6 krb5kdc[6279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:07 dist-test-slave-1nl6 krb5kdc[6279](info): set up 2 sockets
Aug 08 17:01:07 dist-test-slave-1nl6 krb5kdc[6279](info): commencing operation
krb5kdc: starting...
W20260808 17:01:08.433122 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:08 dist-test-slave-1nl6 krb5kdc[6279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:01:08.551867 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38055
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:08.632850 6296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:08.633040 6296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:08.633062 6296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:08.634099 6296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:08.634135 6296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:08.634147 6296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:08.634159 6296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:08.634189 6296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:08.635607 6296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38055
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:08.635790 6296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:08.635954 6296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:08.640617 6310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:08.640630 6307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:08.640666 6296 server_base.cc:1034] running on GCE node
W20260808 17:01:08.640630 6308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:08.641104 6296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:08.641404 6296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:08.642570 6296 hybrid_clock.cc:648] HybridClock initialized: now 1786208468642553 us; error 29 us; skew 500 ppm
Aug 08 17:01:08 dist-test-slave-1nl6 krb5kdc[6279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208468, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:08.645129 6296 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:01:08.645552 6296 webserver.cc:460] Webserver started at http://127.0.0.1:40715/ using document root <none> and password file <none>
I20260808 17:01:08.645779 6296 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:08.645852 6296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:08.646044 6296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:08.647149 6296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "851d6a230fc94650badd4accc96eb8af"
format_stamp: "Formatted at 2026-08-08 17:01:08 on dist-test-slave-1nl6"
server_key: "68ffd2a82c1578aac726e5578a117fe7"
server_key_iv: "de3ea0773785ee8000e964feaca210e6"
server_key_version: "encryptionkey@0"
I20260808 17:01:08.647531 6296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "851d6a230fc94650badd4accc96eb8af"
format_stamp: "Formatted at 2026-08-08 17:01:08 on dist-test-slave-1nl6"
server_key: "68ffd2a82c1578aac726e5578a117fe7"
server_key_iv: "de3ea0773785ee8000e964feaca210e6"
server_key_version: "encryptionkey@0"
I20260808 17:01:08.648842 6296 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:08.649538 6320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:08.649711 6296 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:08.649821 6296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "851d6a230fc94650badd4accc96eb8af"
format_stamp: "Formatted at 2026-08-08 17:01:08 on dist-test-slave-1nl6"
server_key: "68ffd2a82c1578aac726e5578a117fe7"
server_key_iv: "de3ea0773785ee8000e964feaca210e6"
server_key_version: "encryptionkey@0"
I20260808 17:01:08.649907 6296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:08.653493 6296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:08.655671 6296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:08.655827 6296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:08.659983 6296 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38055
I20260808 17:01:08.660010 6390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38055 every 8 connection(s)
I20260808 17:01:08.660341 6296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:08.661312 6391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:08.663499 6391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af: Bootstrap starting.
I20260808 17:01:08.664011 6391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:08.664252 6391 log.cc:824] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:08.664897 6391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af: No bootstrap required, opened a new log
I20260808 17:01:08.666409 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 6296
I20260808 17:01:08.666532 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:08.666731 8249 external_mini_cluster.cc:1428] Setting key 42d5f882063f5280ed0ccf7da03b55cd
I20260808 17:01:08.666611 6391 raft_consensus.cc:348] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "851d6a230fc94650badd4accc96eb8af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38055 } }
I20260808 17:01:08.666776 6391 raft_consensus.cc:374] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:08.666813 6391 raft_consensus.cc:729] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 851d6a230fc94650badd4accc96eb8af, State: Initialized, Role: FOLLOWER
I20260808 17:01:08.666980 6391 consensus_queue.cc:260] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "851d6a230fc94650badd4accc96eb8af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38055 } }
I20260808 17:01:08.667074 6391 raft_consensus.cc:388] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:08.667137 6391 raft_consensus.cc:482] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:08.667192 6391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:08.667805 6391 raft_consensus.cc:504] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "851d6a230fc94650badd4accc96eb8af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38055 } }
I20260808 17:01:08.667930 6391 leader_election.cc:302] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 851d6a230fc94650badd4accc96eb8af; no voters:
I20260808 17:01:08.668141 6397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:08.668267 6397 raft_consensus.cc:686] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [term 1 LEADER]: Becoming Leader. State: Replica: 851d6a230fc94650badd4accc96eb8af, State: Running, Role: LEADER
I20260808 17:01:08.668421 6391 leader_election.cc:288] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:08.668406 6397 consensus_queue.cc:237] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "851d6a230fc94650badd4accc96eb8af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38055 } }
I20260808 17:01:08.668817 6397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "851d6a230fc94650badd4accc96eb8af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "851d6a230fc94650badd4accc96eb8af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38055 } } }
I20260808 17:01:08.668799 6391 sys_catalog.cc:564] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:08.668943 6397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:08.668831 6398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 851d6a230fc94650badd4accc96eb8af. Latest consensus state: current_term: 1 leader_uuid: "851d6a230fc94650badd4accc96eb8af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "851d6a230fc94650badd4accc96eb8af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38055 } } }
I20260808 17:01:08.669060 6398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:08.669459 6396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:08.667211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45055 (local address 127.0.0.1:38055)
0808 17:01:08.667415 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:08.667424 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:08.667461 (+ 37us) server_negotiation.cc:407] Connection header received
0808 17:01:08.667767 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:08.667772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:08.667795 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:08.667950 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:08.668651 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:08.668655 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:08.668655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:08.668659 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:08.668707 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:08.668720 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:08.668807 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:08.669177 (+ 370us) server_negotiation.cc:299] Negotiation successful
0808 17:01:08.669223 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":103,"threads_started":1}
I20260808 17:01:08.670686 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 17:01:08.671142 6421 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:01:08.671190 6421 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:01:08.671274 6406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:08.671487 6406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:08.673337 6396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:08.672277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50637 (local address 127.0.0.1:38055)
0808 17:01:08.672322 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:08.672324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:08.672473 (+ 149us) server_negotiation.cc:407] Connection header received
0808 17:01:08.672519 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:08.672520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:08.672525 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:08.672584 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:08.673124 (+ 540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:08.673125 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:08.673126 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:08.673130 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:08.673144 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:08.673148 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:08.673205 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:08.673260 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 17:01:08.673275 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 17:01:08.673493 6406 catalog_manager.cc:1347] Generated new cluster ID: 1b706d69f8b443179ba356ea005be49e
I20260808 17:01:08.673544 6406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:08.704788 6406 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:08.704972 6406 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:08.715878 6406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 851d6a230fc94650badd4accc96eb8af: Generated new TSK 0
I20260808 17:01:08.716075 6406 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:08.726001 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 6296
2026-08-08T17:01:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1510 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:08 dist-test-slave-1nl6 krb5kdc[6434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:08 dist-test-slave-1nl6 krb5kdc[6434](info): set up 2 sockets
Aug 08 17:01:08 dist-test-slave-1nl6 krb5kdc[6434](info): commencing operation
krb5kdc: starting...
W20260808 17:01:09.950125 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:10 dist-test-slave-1nl6 krb5kdc[6434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:01:10.079326 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:10.162175 6455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:10.162376 6455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:10.162398 6455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:10.163610 6455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:10.163655 6455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:10.163669 6455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:10.163682 6455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:10.163728 6455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:10.165184 6455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:10.165418 6455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:10.165632 6455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:10.170392 6467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:10.170367 6464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:10.170418 6463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:10.170925 6455 server_base.cc:1034] running on GCE node
I20260808 17:01:10.171149 6455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:10.171451 6455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:10.172593 6455 hybrid_clock.cc:648] HybridClock initialized: now 1786208470172578 us; error 32 us; skew 500 ppm
Aug 08 17:01:10 dist-test-slave-1nl6 krb5kdc[6434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208470, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:10.175035 6455 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:01:10.175463 6455 webserver.cc:460] Webserver started at http://127.0.0.1:45405/ using document root <none> and password file <none>
I20260808 17:01:10.175700 6455 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:10.175768 6455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:10.175904 6455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:10.176928 6455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "b7f3902a12ca40d79ef030a9e5aab66f"
format_stamp: "Formatted at 2026-08-08 17:01:10 on dist-test-slave-1nl6"
server_key: "4626c87bd4cb56189cb99d0054abd99b"
server_key_iv: "a7c0b343714e9ada3e84fbee76a56de0"
server_key_version: "encryptionkey@0"
I20260808 17:01:10.177307 6455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "b7f3902a12ca40d79ef030a9e5aab66f"
format_stamp: "Formatted at 2026-08-08 17:01:10 on dist-test-slave-1nl6"
server_key: "4626c87bd4cb56189cb99d0054abd99b"
server_key_iv: "a7c0b343714e9ada3e84fbee76a56de0"
server_key_version: "encryptionkey@0"
I20260808 17:01:10.178622 6455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 17:01:10.179402 6476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:10.179572 6455 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:10.179672 6455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "b7f3902a12ca40d79ef030a9e5aab66f"
format_stamp: "Formatted at 2026-08-08 17:01:10 on dist-test-slave-1nl6"
server_key: "4626c87bd4cb56189cb99d0054abd99b"
server_key_iv: "a7c0b343714e9ada3e84fbee76a56de0"
server_key_version: "encryptionkey@0"
I20260808 17:01:10.179754 6455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:10.189816 6455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:10.191636 6455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:10.191787 6455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:10.195796 6455 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39853
I20260808 17:01:10.195832 6550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39853 every 8 connection(s)
I20260808 17:01:10.196166 6455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:10.197147 6551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:10.199362 6551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f: Bootstrap starting.
I20260808 17:01:10.199888 6551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:10.200116 6551 log.cc:824] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:10.200737 6551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f: No bootstrap required, opened a new log
I20260808 17:01:10.202206 6551 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f3902a12ca40d79ef030a9e5aab66f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39853 } }
I20260808 17:01:10.202333 6551 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:10.202358 6551 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7f3902a12ca40d79ef030a9e5aab66f, State: Initialized, Role: FOLLOWER
I20260808 17:01:10.202484 6551 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f3902a12ca40d79ef030a9e5aab66f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39853 } }
I20260808 17:01:10.202565 6551 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:10.202587 6551 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:10.202620 6551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:10.203203 6551 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f3902a12ca40d79ef030a9e5aab66f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39853 } }
I20260808 17:01:10.203313 6551 leader_election.cc:302] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7f3902a12ca40d79ef030a9e5aab66f; no voters:
I20260808 17:01:10.203480 6551 leader_election.cc:288] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:10.203509 6555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:10.203564 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 6455
I20260808 17:01:10.203678 6555 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [term 1 LEADER]: Becoming Leader. State: Replica: b7f3902a12ca40d79ef030a9e5aab66f, State: Running, Role: LEADER
I20260808 17:01:10.203691 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:10.203758 6551 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:10.203786 6555 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f3902a12ca40d79ef030a9e5aab66f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39853 } }
I20260808 17:01:10.203883 8249 external_mini_cluster.cc:1428] Setting key 6c0ce251fee17c32b693b72a7e81f3b1
I20260808 17:01:10.204125 6555 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7f3902a12ca40d79ef030a9e5aab66f. Latest consensus state: current_term: 1 leader_uuid: "b7f3902a12ca40d79ef030a9e5aab66f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f3902a12ca40d79ef030a9e5aab66f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39853 } } }
I20260808 17:01:10.204212 6555 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:10.204327 6556 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7f3902a12ca40d79ef030a9e5aab66f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7f3902a12ca40d79ef030a9e5aab66f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39853 } } }
I20260808 17:01:10.204416 6556 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:10.204447 6562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:10.204962 6562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:10.206795 6562 catalog_manager.cc:1347] Generated new cluster ID: 60fddaa2e38b4d71be612d00e7977972
I20260808 17:01:10.206854 6562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:10.211771 6562 catalog_manager.cc:1370] Generated new certificate authority record
Aug 08 17:01:10 dist-test-slave-1nl6 krb5kdc[6434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 17:01:10.212320 6562 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:10.216198 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:10.204524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36267 (local address 127.0.0.1:39853)
0808 17:01:10.204711 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:10.204721 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:10.205006 (+ 285us) server_negotiation.cc:407] Connection header received
0808 17:01:10.205201 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:10.205209 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:10.205265 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:10.205416 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:10.206517 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:10.207884 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:10.208940 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:10.209583 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:10.209692 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:10.212691 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:10.212695 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:10.212695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:10.212706 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:10.214626 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:10.215047 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:10.215050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:10.215050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:10.215150 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:10.215360 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:10.215361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:10.215361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:10.215610 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:10.215742 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:10.215852 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 17:01:10.215989 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260808 17:01:10.217331 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:10.224123 6562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7f3902a12ca40d79ef030a9e5aab66f: Generated new TSK 0
I20260808 17:01:10.224308 6562 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:10.225883 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:10.218757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54871 (local address 127.0.0.1:39853)
0808 17:01:10.218795 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:10.218796 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:10.219087 (+ 291us) server_negotiation.cc:407] Connection header received
0808 17:01:10.219147 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:10.219148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:10.219158 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:10.219213 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:10.219820 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:10.220525 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:10.221617 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:10.221908 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:10.221979 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:10.223295 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:10.223297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:10.223297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:10.223304 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:10.224868 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:10.225238 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:10.225239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:10.225239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:10.225288 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:10.225485 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:10.225486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:10.225486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:10.225572 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:10.225636 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:10.225766 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 17:01:10.225832 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:01:10.227712 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 6455
2026-08-08T17:01:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1499 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:10 dist-test-slave-1nl6 krb5kdc[6594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:10 dist-test-slave-1nl6 krb5kdc[6594](info): set up 2 sockets
Aug 08 17:01:10 dist-test-slave-1nl6 krb5kdc[6594](info): commencing operation
krb5kdc: starting...
W20260808 17:01:11.431417 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:11 dist-test-slave-1nl6 krb5kdc[6594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:01:11.553158 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33041
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:11.634802 6609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:11.635047 6609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:11.635088 6609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:11.636216 6609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:11.636281 6609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:11.636305 6609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:11.636325 6609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:11.636356 6609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:11.637789 6609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:11.638058 6609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:11.638279 6609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:11.643080 6619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:11.643103 6617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:11.643160 6609 server_base.cc:1034] running on GCE node
W20260808 17:01:11.643129 6616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:11.643535 6609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:11.643841 6609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:11.644996 6609 hybrid_clock.cc:648] HybridClock initialized: now 1786208471644991 us; error 33 us; skew 500 ppm
Aug 08 17:01:11 dist-test-slave-1nl6 krb5kdc[6594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208471, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:11.652621 6609 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:01:11.653157 6609 webserver.cc:460] Webserver started at http://127.0.0.1:41693/ using document root <none> and password file <none>
I20260808 17:01:11.653483 6609 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:11.653554 6609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:11.653695 6609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:11.654812 6609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "e6aa3f3b257c424f80bd8c4845a5198e"
format_stamp: "Formatted at 2026-08-08 17:01:11 on dist-test-slave-1nl6"
server_key: "0ea32fae6f9c90bb8b5cecb12f1c371f"
server_key_iv: "37b4fc4bda1890753027f7363bead767"
server_key_version: "encryptionkey@0"
I20260808 17:01:11.655227 6609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "e6aa3f3b257c424f80bd8c4845a5198e"
format_stamp: "Formatted at 2026-08-08 17:01:11 on dist-test-slave-1nl6"
server_key: "0ea32fae6f9c90bb8b5cecb12f1c371f"
server_key_iv: "37b4fc4bda1890753027f7363bead767"
server_key_version: "encryptionkey@0"
I20260808 17:01:11.656544 6609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:11.657261 6631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:11.657433 6609 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:11.657533 6609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "e6aa3f3b257c424f80bd8c4845a5198e"
format_stamp: "Formatted at 2026-08-08 17:01:11 on dist-test-slave-1nl6"
server_key: "0ea32fae6f9c90bb8b5cecb12f1c371f"
server_key_iv: "37b4fc4bda1890753027f7363bead767"
server_key_version: "encryptionkey@0"
I20260808 17:01:11.657616 6609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:11.685391 6609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:11.687422 6609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:11.687582 6609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:11.691622 6609 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33041
I20260808 17:01:11.691674 6704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33041 every 8 connection(s)
I20260808 17:01:11.692030 6609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:11.693017 6705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:11.695338 6705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e: Bootstrap starting.
I20260808 17:01:11.695919 6705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:11.696214 6705 log.cc:824] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:11.696892 6705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e: No bootstrap required, opened a new log
I20260808 17:01:11.698450 6705 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6aa3f3b257c424f80bd8c4845a5198e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } }
I20260808 17:01:11.698591 6705 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:11.698652 6705 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6aa3f3b257c424f80bd8c4845a5198e, State: Initialized, Role: FOLLOWER
I20260808 17:01:11.698813 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 6609
I20260808 17:01:11.698804 6705 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6aa3f3b257c424f80bd8c4845a5198e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } }
I20260808 17:01:11.698932 6705 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:11.698982 6705 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:11.698958 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:11.699023 6705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:11.699178 8249 external_mini_cluster.cc:1428] Setting key 2489058445b6ba91a176c69b05361d35
I20260808 17:01:11.699666 6705 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6aa3f3b257c424f80bd8c4845a5198e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } }
I20260808 17:01:11.699788 6705 leader_election.cc:302] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6aa3f3b257c424f80bd8c4845a5198e; no voters:
I20260808 17:01:11.699992 6705 leader_election.cc:288] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:11.700022 6710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:11.700135 6710 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [term 1 LEADER]: Becoming Leader. State: Replica: e6aa3f3b257c424f80bd8c4845a5198e, State: Running, Role: LEADER
I20260808 17:01:11.700224 6710 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6aa3f3b257c424f80bd8c4845a5198e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } }
I20260808 17:01:11.700268 6705 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:11.700554 6710 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6aa3f3b257c424f80bd8c4845a5198e. Latest consensus state: current_term: 1 leader_uuid: "e6aa3f3b257c424f80bd8c4845a5198e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6aa3f3b257c424f80bd8c4845a5198e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } } }
I20260808 17:01:11.700631 6710 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:11.700800 6713 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6aa3f3b257c424f80bd8c4845a5198e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6aa3f3b257c424f80bd8c4845a5198e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } } }
I20260808 17:01:11.700886 6713 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:11.701017 6717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:11.701575 6717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:11.703301 6717 catalog_manager.cc:1347] Generated new cluster ID: cc917af008644c398d8645544940a6ee
I20260808 17:01:11.703361 6717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:11.705667 6709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:11.699655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54765 (local address 127.0.0.1:33041)
0808 17:01:11.699874 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:11.699884 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:11.699919 (+ 35us) server_negotiation.cc:407] Connection header received
0808 17:01:11.700026 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:11.700031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:11.700103 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:11.700525 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:11.700625 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:11.702495 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:11.703584 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:11.704192 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:11.704292 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:11.704996 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:11.705001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:11.705001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:11.705195 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:11.705231 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:11.705244 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:11.705303 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:11.705407 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 17:01:11.705469 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":103,"threads_started":1}
I20260808 17:01:11.706580 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:11.711509 6709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:11.708112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38567 (local address 127.0.0.1:33041)
0808 17:01:11.708148 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:11.708150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:11.708284 (+ 134us) server_negotiation.cc:407] Connection header received
0808 17:01:11.708341 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:11.708342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:11.708351 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:11.708396 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:11.708553 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:11.709271 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:11.710250 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:11.710547 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:11.710608 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:11.711251 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:11.711252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:11.711252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:11.711257 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:11.711284 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:11.711288 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:11.711338 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:11.711419 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 17:01:11.711441 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:01:11.714967 6717 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:11.715435 6717 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:11.720723 6717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6aa3f3b257c424f80bd8c4845a5198e: Generated new TSK 0
I20260808 17:01:11.720898 6717 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:11.723384 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 6609
2026-08-08T17:01:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1501 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:11 dist-test-slave-1nl6 krb5kdc[6748](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:11 dist-test-slave-1nl6 krb5kdc[6748](info): set up 2 sockets
Aug 08 17:01:11 dist-test-slave-1nl6 krb5kdc[6748](info): commencing operation
krb5kdc: starting...
W20260808 17:01:12.940179 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:13 dist-test-slave-1nl6 krb5kdc[6748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:01:13.056130 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38899
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:13.137574 6763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:13.137775 6763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:13.137799 6763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:13.138986 6763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:13.139029 6763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:13.139043 6763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:13.139083 6763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:13.139107 6763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:13.140653 6763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:13.140904 6763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:13.141148 6763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:13.145857 6776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:13.145867 6774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:13.146054 6763 server_base.cc:1034] running on GCE node
W20260808 17:01:13.145902 6773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:13.146351 6763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:13.146636 6763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:13.147775 6763 hybrid_clock.cc:648] HybridClock initialized: now 1786208473147754 us; error 42 us; skew 500 ppm
Aug 08 17:01:13 dist-test-slave-1nl6 krb5kdc[6748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208473, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:13.150171 6763 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:01:13.150625 6763 webserver.cc:460] Webserver started at http://127.0.0.1:43299/ using document root <none> and password file <none>
I20260808 17:01:13.150862 6763 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:13.150951 6763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:13.151049 6763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:13.152055 6763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "cf4d9e21ab304173aef3bf0d93032233"
format_stamp: "Formatted at 2026-08-08 17:01:13 on dist-test-slave-1nl6"
server_key: "02dd0117714028ff803f1fbc0ea0ef3f"
server_key_iv: "04977d25e3094f0ffb650923b3872226"
server_key_version: "encryptionkey@0"
I20260808 17:01:13.152413 6763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "cf4d9e21ab304173aef3bf0d93032233"
format_stamp: "Formatted at 2026-08-08 17:01:13 on dist-test-slave-1nl6"
server_key: "02dd0117714028ff803f1fbc0ea0ef3f"
server_key_iv: "04977d25e3094f0ffb650923b3872226"
server_key_version: "encryptionkey@0"
I20260808 17:01:13.153756 6763 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:01:13.154489 6784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:13.154655 6763 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:01:13.154757 6763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "cf4d9e21ab304173aef3bf0d93032233"
format_stamp: "Formatted at 2026-08-08 17:01:13 on dist-test-slave-1nl6"
server_key: "02dd0117714028ff803f1fbc0ea0ef3f"
server_key_iv: "04977d25e3094f0ffb650923b3872226"
server_key_version: "encryptionkey@0"
I20260808 17:01:13.154827 6763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:13.158036 6763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:13.159812 6763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:13.159968 6763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:13.164108 6763 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38899
I20260808 17:01:13.164139 6862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38899 every 8 connection(s)
I20260808 17:01:13.164451 6763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:13.165388 6863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:13.167646 6863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233: Bootstrap starting.
I20260808 17:01:13.168159 6863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:13.168403 6863 log.cc:824] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:13.169016 6863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233: No bootstrap required, opened a new log
I20260808 17:01:13.170586 6863 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf4d9e21ab304173aef3bf0d93032233" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38899 } }
I20260808 17:01:13.170713 6863 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:13.170693 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 6763
I20260808 17:01:13.170734 6863 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf4d9e21ab304173aef3bf0d93032233, State: Initialized, Role: FOLLOWER
I20260808 17:01:13.170830 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:13.170921 6863 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf4d9e21ab304173aef3bf0d93032233" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38899 } }
I20260808 17:01:13.171008 6863 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:13.171032 6863 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:13.171070 8249 external_mini_cluster.cc:1428] Setting key 28f72b3d5b6a02d5aa153596248ac515
I20260808 17:01:13.171103 6863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:13.172196 6863 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf4d9e21ab304173aef3bf0d93032233" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38899 } }
I20260808 17:01:13.172593 6863 leader_election.cc:302] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf4d9e21ab304173aef3bf0d93032233; no voters:
I20260808 17:01:13.172778 6863 leader_election.cc:288] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:13.172806 6868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:13.173002 6868 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [term 1 LEADER]: Becoming Leader. State: Replica: cf4d9e21ab304173aef3bf0d93032233, State: Running, Role: LEADER
I20260808 17:01:13.173066 6863 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:13.173127 6868 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf4d9e21ab304173aef3bf0d93032233" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38899 } }
I20260808 17:01:13.173393 6867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:13.171542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49947 (local address 127.0.0.1:38899)
0808 17:01:13.171766 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:13.171779 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:13.171818 (+ 39us) server_negotiation.cc:407] Connection header received
0808 17:01:13.171946 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:13.171953 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:13.171999 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:13.172156 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:13.172718 (+ 562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:13.172722 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:13.172723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:13.172728 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:13.172800 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:13.172818 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:13.173114 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:13.173149 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 17:01:13.173189 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":106,"threads_started":1}
I20260808 17:01:13.173552 6870 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf4d9e21ab304173aef3bf0d93032233" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf4d9e21ab304173aef3bf0d93032233" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38899 } } }
I20260808 17:01:13.173643 6870 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:13.173806 6871 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf4d9e21ab304173aef3bf0d93032233. Latest consensus state: current_term: 1 leader_uuid: "cf4d9e21ab304173aef3bf0d93032233" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf4d9e21ab304173aef3bf0d93032233" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38899 } } }
I20260808 17:01:13.173916 6871 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:13.173851 6876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:13.174343 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:13.174410 6876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:13.177052 6867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:13.175953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46227 (local address 127.0.0.1:38899)
0808 17:01:13.175984 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:13.175986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:13.176139 (+ 153us) server_negotiation.cc:407] Connection header received
0808 17:01:13.176214 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:13.176215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:13.176231 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:13.176286 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:13.176844 (+ 558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:13.176845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:13.176846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:13.176850 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:13.176862 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:13.176866 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:13.176922 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:13.176977 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 17:01:13.176991 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260808 17:01:13.179939 6876 catalog_manager.cc:1347] Generated new cluster ID: 411391e481b541cbb6bda4cf81dcecc2
I20260808 17:01:13.179984 6876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:13.192206 6876 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:13.192363 6876 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:13.198268 6876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf4d9e21ab304173aef3bf0d93032233: Generated new TSK 0
I20260808 17:01:13.198433 6876 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:13.205806 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 6763
2026-08-08T17:01:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1477 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:13 dist-test-slave-1nl6 krb5kdc[6908](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:13 dist-test-slave-1nl6 krb5kdc[6908](info): set up 2 sockets
Aug 08 17:01:13 dist-test-slave-1nl6 krb5kdc[6908](info): commencing operation
krb5kdc: starting...
W20260808 17:01:14.415964 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:14 dist-test-slave-1nl6 krb5kdc[6908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:01:14.537910 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39611
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:14.618016 6926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:14.618202 6926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:14.618224 6926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:14.619292 6926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:14.619331 6926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:14.619345 6926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:14.619359 6926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:14.619370 6926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:14.620740 6926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:14.620939 6926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:14.621093 6926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:14.625716 6932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:14.625735 6935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:14.625905 6926 server_base.cc:1034] running on GCE node
W20260808 17:01:14.625743 6937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:14.626230 6926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:14.626523 6926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:14.627682 6926 hybrid_clock.cc:648] HybridClock initialized: now 1786208474627662 us; error 32 us; skew 500 ppm
Aug 08 17:01:14 dist-test-slave-1nl6 krb5kdc[6908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208474, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:14.630043 6926 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:01:14.630445 6926 webserver.cc:460] Webserver started at http://127.0.0.1:39443/ using document root <none> and password file <none>
I20260808 17:01:14.630658 6926 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:14.630722 6926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:14.630869 6926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:14.631871 6926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "720efa03c98d49f1b54f042890fdc6dc"
format_stamp: "Formatted at 2026-08-08 17:01:14 on dist-test-slave-1nl6"
server_key: "1bbe296d419b108212793e407a436a55"
server_key_iv: "252f6f6afb0c69b1b3cb9ebcd3246157"
server_key_version: "encryptionkey@0"
I20260808 17:01:14.632256 6926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "720efa03c98d49f1b54f042890fdc6dc"
format_stamp: "Formatted at 2026-08-08 17:01:14 on dist-test-slave-1nl6"
server_key: "1bbe296d419b108212793e407a436a55"
server_key_iv: "252f6f6afb0c69b1b3cb9ebcd3246157"
server_key_version: "encryptionkey@0"
I20260808 17:01:14.633606 6926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 17:01:14.634279 6949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:14.634450 6926 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:01:14.634550 6926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "720efa03c98d49f1b54f042890fdc6dc"
format_stamp: "Formatted at 2026-08-08 17:01:14 on dist-test-slave-1nl6"
server_key: "1bbe296d419b108212793e407a436a55"
server_key_iv: "252f6f6afb0c69b1b3cb9ebcd3246157"
server_key_version: "encryptionkey@0"
I20260808 17:01:14.634629 6926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:14.648998 6926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:14.650660 6926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:14.650812 6926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:14.654678 6926 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39611
I20260808 17:01:14.654719 7029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39611 every 8 connection(s)
I20260808 17:01:14.655063 6926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:14.655994 7031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:14.658106 7031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc: Bootstrap starting.
I20260808 17:01:14.658632 7031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:14.658860 7031 log.cc:824] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:14.659577 7031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc: No bootstrap required, opened a new log
I20260808 17:01:14.661075 7031 raft_consensus.cc:348] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "720efa03c98d49f1b54f042890fdc6dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39611 } }
I20260808 17:01:14.661209 7031 raft_consensus.cc:374] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:14.661232 7031 raft_consensus.cc:729] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 720efa03c98d49f1b54f042890fdc6dc, State: Initialized, Role: FOLLOWER
I20260808 17:01:14.661376 7031 consensus_queue.cc:260] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "720efa03c98d49f1b54f042890fdc6dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39611 } }
I20260808 17:01:14.661460 7031 raft_consensus.cc:388] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:14.661484 7031 raft_consensus.cc:482] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:14.661517 7031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:14.661983 7031 raft_consensus.cc:504] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "720efa03c98d49f1b54f042890fdc6dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39611 } }
I20260808 17:01:14.662065 7031 leader_election.cc:302] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 720efa03c98d49f1b54f042890fdc6dc; no voters:
I20260808 17:01:14.662180 7031 leader_election.cc:288] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:14.662252 7034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:14.662422 7031 sys_catalog.cc:564] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:14.662649 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 6926
I20260808 17:01:14.662760 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:14.663003 8249 external_mini_cluster.cc:1428] Setting key 319403476bb13aa83853146a5069407f
I20260808 17:01:14.662436 7034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [term 1 LEADER]: Becoming Leader. State: Replica: 720efa03c98d49f1b54f042890fdc6dc, State: Running, Role: LEADER
I20260808 17:01:14.663254 7034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "720efa03c98d49f1b54f042890fdc6dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39611 } }
I20260808 17:01:14.664191 7036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "720efa03c98d49f1b54f042890fdc6dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "720efa03c98d49f1b54f042890fdc6dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39611 } } }
I20260808 17:01:14.664283 7036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:14.664469 7037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 720efa03c98d49f1b54f042890fdc6dc. Latest consensus state: current_term: 1 leader_uuid: "720efa03c98d49f1b54f042890fdc6dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "720efa03c98d49f1b54f042890fdc6dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39611 } } }
I20260808 17:01:14.664538 7037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc [sys.catalog]: This master's current role is: LEADER
W20260808 17:01:14.664580 7054 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:01:14.664698 7054 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:01:14.664858 7059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:14.665035 7059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:14.666478 7059 catalog_manager.cc:1347] Generated new cluster ID: dc1839381af14771bc6ee138a5d69111
I20260808 17:01:14.666528 7059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:01:14 dist-test-slave-1nl6 krb5kdc[6908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 17:01:14.675560 7047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:14.663492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:55273 (local address 127.0.0.1:39611)
0808 17:01:14.664147 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:14.664158 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:14.664195 (+ 37us) server_negotiation.cc:407] Connection header received
0808 17:01:14.664332 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:14.664339 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:14.664403 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:14.664565 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:14.665331 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:14.666649 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:14.667693 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:14.668269 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:14.668376 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:14.671199 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:14.671203 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:14.671204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:14.671215 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:14.673859 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:14.674282 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:14.674285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:14.674285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:14.674412 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:14.674637 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:14.674638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:14.674639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:14.674979 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:14.675137 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:14.675242 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 17:01:14.675353 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":76,"threads_started":1}
I20260808 17:01:14.676487 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:14.684990 7047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:14.678013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:48851 (local address 127.0.0.1:39611)
0808 17:01:14.678045 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:14.678046 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:14.678055 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:01:14.678132 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:14.678133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:14.678141 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:14.678191 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:14.678800 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:14.679523 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:14.680561 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:14.680863 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:14.680918 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:14.682251 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:14.682252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:14.682252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:14.682259 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:14.683793 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:14.684200 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:14.684201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:14.684201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:14.684266 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:14.684506 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:14.684507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:14.684507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:14.684618 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:14.684707 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:14.684849 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 17:01:14.684907 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 17:01:14.696919 7059 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:14.697383 7059 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:14.706123 7059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 720efa03c98d49f1b54f042890fdc6dc: Generated new TSK 0
I20260808 17:01:14.706291 7059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:14.717087 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 6926
W20260808 17:01:14.734133 6905 connection.cc:535] client connection to 127.0.0.1:39611 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39611): Connection reset by peer (error 104)
2026-08-08T17:01:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:14 dist-test-slave-1nl6 krb5kdc[7074](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:14 dist-test-slave-1nl6 krb5kdc[7074](info): set up 2 sockets
Aug 08 17:01:14 dist-test-slave-1nl6 krb5kdc[7074](info): commencing operation
krb5kdc: starting...
W20260808 17:01:15.933619 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:16 dist-test-slave-1nl6 krb5kdc[7074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:01:16.049526 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39395
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:16.130102 7094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:16.130293 7094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:16.130316 7094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:16.131402 7094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:16.131436 7094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:16.131450 7094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:16.131462 7094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:16.131475 7094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:16.132871 7094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:16.133055 7094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:16.133215 7094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:16.137977 7103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:16.138209 7094 server_base.cc:1034] running on GCE node
W20260808 17:01:16.137988 7102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:16.138015 7106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:16.138701 7094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:16.139065 7094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:16.140234 7094 hybrid_clock.cc:648] HybridClock initialized: now 1786208476140213 us; error 34 us; skew 500 ppm
Aug 08 17:01:16 dist-test-slave-1nl6 krb5kdc[7074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208476, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:16.143596 7094 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:01:16.144091 7094 webserver.cc:460] Webserver started at http://127.0.0.1:43819/ using document root <none> and password file <none>
I20260808 17:01:16.144383 7094 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:16.144496 7094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:16.144641 7094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:16.146091 7094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "787da09f4f6c45cb83ac35ee50d49b92"
format_stamp: "Formatted at 2026-08-08 17:01:16 on dist-test-slave-1nl6"
server_key: "fcb0e018db869175a98785a87bda74c8"
server_key_iv: "d380044c7713d23f3531f0daa99f0931"
server_key_version: "encryptionkey@0"
I20260808 17:01:16.146559 7094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "787da09f4f6c45cb83ac35ee50d49b92"
format_stamp: "Formatted at 2026-08-08 17:01:16 on dist-test-slave-1nl6"
server_key: "fcb0e018db869175a98785a87bda74c8"
server_key_iv: "d380044c7713d23f3531f0daa99f0931"
server_key_version: "encryptionkey@0"
I20260808 17:01:16.148095 7094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:01:16.148851 7113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:16.149010 7094 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:16.149080 7094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "787da09f4f6c45cb83ac35ee50d49b92"
format_stamp: "Formatted at 2026-08-08 17:01:16 on dist-test-slave-1nl6"
server_key: "fcb0e018db869175a98785a87bda74c8"
server_key_iv: "d380044c7713d23f3531f0daa99f0931"
server_key_version: "encryptionkey@0"
I20260808 17:01:16.149148 7094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:16.156075 7094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:16.157783 7094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:16.157927 7094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:16.161806 7094 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39395
I20260808 17:01:16.161870 7188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39395 every 8 connection(s)
I20260808 17:01:16.162186 7094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:16.163232 7189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:16.163818 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 7094
I20260808 17:01:16.163937 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:16.164148 8249 external_mini_cluster.cc:1428] Setting key d69aca32f1acbb5f83adaf8251f05ee2
I20260808 17:01:16.165940 7189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92: Bootstrap starting.
I20260808 17:01:16.166510 7189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:16.166797 7189 log.cc:824] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:16.167629 7189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92: No bootstrap required, opened a new log
I20260808 17:01:16.169775 7189 raft_consensus.cc:348] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787da09f4f6c45cb83ac35ee50d49b92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39395 } }
I20260808 17:01:16.169937 7189 raft_consensus.cc:374] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:16.169971 7189 raft_consensus.cc:729] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 787da09f4f6c45cb83ac35ee50d49b92, State: Initialized, Role: FOLLOWER
I20260808 17:01:16.170112 7189 consensus_queue.cc:260] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787da09f4f6c45cb83ac35ee50d49b92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39395 } }
I20260808 17:01:16.170226 7189 raft_consensus.cc:388] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:16.170275 7189 raft_consensus.cc:482] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:16.170349 7189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:16.170979 7189 raft_consensus.cc:504] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787da09f4f6c45cb83ac35ee50d49b92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39395 } }
I20260808 17:01:16.171119 7189 leader_election.cc:302] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 787da09f4f6c45cb83ac35ee50d49b92; no voters:
I20260808 17:01:16.171307 7189 leader_election.cc:288] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:16.171347 7195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:16.171537 7195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [term 1 LEADER]: Becoming Leader. State: Replica: 787da09f4f6c45cb83ac35ee50d49b92, State: Running, Role: LEADER
I20260808 17:01:16.171574 7189 sys_catalog.cc:564] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:16.171700 7195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787da09f4f6c45cb83ac35ee50d49b92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39395 } }
I20260808 17:01:16.172049 7195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 787da09f4f6c45cb83ac35ee50d49b92. Latest consensus state: current_term: 1 leader_uuid: "787da09f4f6c45cb83ac35ee50d49b92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787da09f4f6c45cb83ac35ee50d49b92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39395 } } }
I20260808 17:01:16.172287 7195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:16.172550 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:16.164693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:57015 (local address 127.0.0.1:39395)
0808 17:01:16.164903 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:16.164912 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:16.167132 (+ 2220us) server_negotiation.cc:407] Connection header received
0808 17:01:16.167653 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:16.167660 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:16.167730 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:16.167869 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:16.168120 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:16.168976 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:16.170164 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:16.170665 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:16.170748 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:16.171601 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:16.171605 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:16.171605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:16.171848 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:16.171890 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:16.171908 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:16.171988 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:16.172288 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 17:01:16.172320 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":72,"threads_started":1}
I20260808 17:01:16.172210 7196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "787da09f4f6c45cb83ac35ee50d49b92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "787da09f4f6c45cb83ac35ee50d49b92" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39395 } } }
I20260808 17:01:16.172679 7196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:16.172842 7207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:16.173316 7207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:16.173698 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:16.174937 7207 catalog_manager.cc:1347] Generated new cluster ID: d1135900478547e98c9fef3e48eda2e2
I20260808 17:01:16.174993 7207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:16.177851 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:16.175065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:45225 (local address 127.0.0.1:39395)
0808 17:01:16.175100 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:16.175102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:16.175206 (+ 104us) server_negotiation.cc:407] Connection header received
0808 17:01:16.175280 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:16.175281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:16.175290 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:16.175352 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:16.175503 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:16.175960 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:16.176666 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:16.176905 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:16.176976 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:16.177605 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:16.177606 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:16.177606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:16.177610 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:16.177623 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:16.177627 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:16.177682 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:16.177742 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 17:01:16.177751 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:01:16.180893 7207 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:16.181350 7207 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:16.194414 7207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 787da09f4f6c45cb83ac35ee50d49b92: Generated new TSK 0
I20260808 17:01:16.194595 7207 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:16.199656 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 7094
W20260808 17:01:16.214126 7072 connection.cc:535] client connection to 127.0.0.1:39395 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39395): Connection reset by peer (error 104)
2026-08-08T17:01:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1479 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:16 dist-test-slave-1nl6 krb5kdc[7232](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:16 dist-test-slave-1nl6 krb5kdc[7232](info): set up 2 sockets
Aug 08 17:01:16 dist-test-slave-1nl6 krb5kdc[7232](info): commencing operation
krb5kdc: starting...
W20260808 17:01:17.420349 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:17 dist-test-slave-1nl6 krb5kdc[7232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:01:17.538625 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45419
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:17.619501 7252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:17.619691 7252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:17.619715 7252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:17.620793 7252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:17.620826 7252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:17.620841 7252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:17.620853 7252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:17.620865 7252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:17.622265 7252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:17.622504 7252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:17.622723 7252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:17.627574 7264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:17.627586 7267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:17.627599 7258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:17.627908 7252 server_base.cc:1034] running on GCE node
I20260808 17:01:17.628126 7252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:17.628358 7252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:17.629500 7252 hybrid_clock.cc:648] HybridClock initialized: now 1786208477629484 us; error 32 us; skew 500 ppm
Aug 08 17:01:17 dist-test-slave-1nl6 krb5kdc[7232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208477, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:17.636899 7252 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:01:17.637316 7252 webserver.cc:460] Webserver started at http://127.0.0.1:40977/ using document root <none> and password file <none>
I20260808 17:01:17.637562 7252 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:17.637609 7252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:17.637761 7252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:17.638787 7252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "ba3602e0c4b84e1d936f762c5370ad90"
format_stamp: "Formatted at 2026-08-08 17:01:17 on dist-test-slave-1nl6"
server_key: "62906997e265198c8e3a8e7b964de408"
server_key_iv: "d12b3cd3c90aa95a2b1e83e84bd20231"
server_key_version: "encryptionkey@0"
I20260808 17:01:17.639253 7252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "ba3602e0c4b84e1d936f762c5370ad90"
format_stamp: "Formatted at 2026-08-08 17:01:17 on dist-test-slave-1nl6"
server_key: "62906997e265198c8e3a8e7b964de408"
server_key_iv: "d12b3cd3c90aa95a2b1e83e84bd20231"
server_key_version: "encryptionkey@0"
I20260808 17:01:17.640622 7252 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:17.641330 7279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:17.641492 7252 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:17.641592 7252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "ba3602e0c4b84e1d936f762c5370ad90"
format_stamp: "Formatted at 2026-08-08 17:01:17 on dist-test-slave-1nl6"
server_key: "62906997e265198c8e3a8e7b964de408"
server_key_iv: "d12b3cd3c90aa95a2b1e83e84bd20231"
server_key_version: "encryptionkey@0"
I20260808 17:01:17.641669 7252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:17.644981 7252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:17.646718 7252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:17.646883 7252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:17.650846 7252 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45419
I20260808 17:01:17.650870 7361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45419 every 8 connection(s)
I20260808 17:01:17.651226 7252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:17.652184 7362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:17.652877 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 7252
I20260808 17:01:17.653008 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:17.653206 8249 external_mini_cluster.cc:1428] Setting key 48ba43bdc84f33a6a410a451bc67ce22
I20260808 17:01:17.655171 7362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90: Bootstrap starting.
I20260808 17:01:17.655576 7367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:17.653719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:60381 (local address 127.0.0.1:45419)
0808 17:01:17.653899 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:17.653908 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:17.653940 (+ 32us) server_negotiation.cc:407] Connection header received
0808 17:01:17.654357 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:17.654363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:17.654386 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:17.654530 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:17.655170 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:17.655173 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:17.655174 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:17.655178 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:17.655237 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:17.655252 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:17.655320 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:17.655381 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 17:01:17.655423 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260808 17:01:17.655926 7362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:17.656261 7362 log.cc:824] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:17.656476 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:17.656956 7362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90: No bootstrap required, opened a new log
I20260808 17:01:17.659222 7362 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3602e0c4b84e1d936f762c5370ad90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45419 } }
I20260808 17:01:17.659408 7362 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:17.659476 7362 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba3602e0c4b84e1d936f762c5370ad90, State: Initialized, Role: FOLLOWER
I20260808 17:01:17.659601 7362 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3602e0c4b84e1d936f762c5370ad90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45419 } }
I20260808 17:01:17.659709 7362 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:17.659754 7362 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:17.659822 7362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:17.660077 7367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:17.657825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:35353 (local address 127.0.0.1:45419)
0808 17:01:17.657862 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:17.657864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:17.659074 (+ 1210us) server_negotiation.cc:407] Connection header received
0808 17:01:17.659138 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:17.659139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:17.659143 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:17.659190 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:17.659845 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:17.659847 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:17.659847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:17.659852 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:17.659867 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:17.659871 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:17.659933 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:17.660006 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 17:01:17.660016 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:01:17.660549 7362 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3602e0c4b84e1d936f762c5370ad90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45419 } }
I20260808 17:01:17.660683 7362 leader_election.cc:302] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba3602e0c4b84e1d936f762c5370ad90; no voters:
I20260808 17:01:17.660887 7362 leader_election.cc:288] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:17.660919 7378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:17.661069 7378 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [term 1 LEADER]: Becoming Leader. State: Replica: ba3602e0c4b84e1d936f762c5370ad90, State: Running, Role: LEADER
I20260808 17:01:17.661121 7362 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:17.661199 7378 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3602e0c4b84e1d936f762c5370ad90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45419 } }
I20260808 17:01:17.661582 7379 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba3602e0c4b84e1d936f762c5370ad90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3602e0c4b84e1d936f762c5370ad90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45419 } } }
I20260808 17:01:17.661672 7379 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:17.661896 7395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:17.661880 7380 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba3602e0c4b84e1d936f762c5370ad90. Latest consensus state: current_term: 1 leader_uuid: "ba3602e0c4b84e1d936f762c5370ad90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba3602e0c4b84e1d936f762c5370ad90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45419 } } }
I20260808 17:01:17.662173 7380 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:17.662458 7395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:17.664072 7395 catalog_manager.cc:1347] Generated new cluster ID: 1a55b24cebcd4cbb9a8bb04216afd072
I20260808 17:01:17.664129 7395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:17.671291 7395 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:17.671450 7395 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:17.674840 7395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba3602e0c4b84e1d936f762c5370ad90: Generated new TSK 0
I20260808 17:01:17.675032 7395 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:17.681167 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 7252
2026-08-08T17:01:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1478 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:17 dist-test-slave-1nl6 krb5kdc[7417](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:17 dist-test-slave-1nl6 krb5kdc[7417](info): set up 2 sockets
Aug 08 17:01:17 dist-test-slave-1nl6 krb5kdc[7417](info): commencing operation
krb5kdc: starting...
W20260808 17:01:18.901964 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:18 dist-test-slave-1nl6 krb5kdc[7417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:01:19.022781 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44659
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:19.106799 7434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:19.107163 7434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:19.107209 7434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:19.108397 7434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:19.108466 7434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:19.108487 7434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:19.108507 7434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:19.108536 7434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:19.110090 7434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44659
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:19.110386 7434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:19.110613 7434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:19.115571 7440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:19.115594 7439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:19.115722 7434 server_base.cc:1034] running on GCE node
W20260808 17:01:19.115583 7442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:19.116060 7434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:19.116379 7434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:19.117543 7434 hybrid_clock.cc:648] HybridClock initialized: now 1786208479117529 us; error 27 us; skew 500 ppm
Aug 08 17:01:19 dist-test-slave-1nl6 krb5kdc[7417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208479, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:19.120443 7434 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:01:19.120869 7434 webserver.cc:460] Webserver started at http://127.0.0.1:34195/ using document root <none> and password file <none>
I20260808 17:01:19.121100 7434 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:19.121146 7434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:19.121297 7434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:19.122377 7434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "d51b319219db4dd9b95941d8bc01572b"
format_stamp: "Formatted at 2026-08-08 17:01:19 on dist-test-slave-1nl6"
server_key: "934a1f1b8c918ca45fc9fdc9f3cf43ce"
server_key_iv: "101ef7328517ce7bec7d977bf84848ba"
server_key_version: "encryptionkey@0"
I20260808 17:01:19.122746 7434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "d51b319219db4dd9b95941d8bc01572b"
format_stamp: "Formatted at 2026-08-08 17:01:19 on dist-test-slave-1nl6"
server_key: "934a1f1b8c918ca45fc9fdc9f3cf43ce"
server_key_iv: "101ef7328517ce7bec7d977bf84848ba"
server_key_version: "encryptionkey@0"
I20260808 17:01:19.124121 7434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:01:19.124862 7454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:19.125046 7434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:19.125144 7434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "d51b319219db4dd9b95941d8bc01572b"
format_stamp: "Formatted at 2026-08-08 17:01:19 on dist-test-slave-1nl6"
server_key: "934a1f1b8c918ca45fc9fdc9f3cf43ce"
server_key_iv: "101ef7328517ce7bec7d977bf84848ba"
server_key_version: "encryptionkey@0"
I20260808 17:01:19.125221 7434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:19.148913 7434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:19.150939 7434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:19.151113 7434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:19.155383 7434 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44659
I20260808 17:01:19.155438 7526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44659 every 8 connection(s)
I20260808 17:01:19.155831 7434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:19.156863 7527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:19.158486 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 7434
I20260808 17:01:19.158588 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:19.158806 8249 external_mini_cluster.cc:1428] Setting key b9603531a6bba68e75e3d7e3d9e569e4
I20260808 17:01:19.159816 7527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b: Bootstrap starting.
I20260808 17:01:19.160555 7527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:19.160888 7527 log.cc:824] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:19.161671 7527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b: No bootstrap required, opened a new log
I20260808 17:01:19.163967 7527 raft_consensus.cc:348] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51b319219db4dd9b95941d8bc01572b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44659 } }
I20260808 17:01:19.164143 7527 raft_consensus.cc:374] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:19.164184 7527 raft_consensus.cc:729] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51b319219db4dd9b95941d8bc01572b, State: Initialized, Role: FOLLOWER
I20260808 17:01:19.164321 7527 consensus_queue.cc:260] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51b319219db4dd9b95941d8bc01572b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44659 } }
I20260808 17:01:19.164417 7527 raft_consensus.cc:388] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:19.164448 7527 raft_consensus.cc:482] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:19.164566 7527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:19.165112 7527 raft_consensus.cc:504] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51b319219db4dd9b95941d8bc01572b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44659 } }
I20260808 17:01:19.165198 7527 leader_election.cc:302] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d51b319219db4dd9b95941d8bc01572b; no voters:
I20260808 17:01:19.165364 7527 leader_election.cc:288] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:19.165411 7533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:19.165549 7533 raft_consensus.cc:686] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [term 1 LEADER]: Becoming Leader. State: Replica: d51b319219db4dd9b95941d8bc01572b, State: Running, Role: LEADER
I20260808 17:01:19.165613 7527 sys_catalog.cc:564] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:19.165702 7533 consensus_queue.cc:237] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51b319219db4dd9b95941d8bc01572b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44659 } }
Aug 08 17:01:19 dist-test-slave-1nl6 krb5kdc[7417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 17:01:19.166078 7533 sys_catalog.cc:455] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d51b319219db4dd9b95941d8bc01572b. Latest consensus state: current_term: 1 leader_uuid: "d51b319219db4dd9b95941d8bc01572b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51b319219db4dd9b95941d8bc01572b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44659 } } }
I20260808 17:01:19.166091 7535 sys_catalog.cc:455] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d51b319219db4dd9b95941d8bc01572b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51b319219db4dd9b95941d8bc01572b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44659 } } }
I20260808 17:01:19.166157 7533 sys_catalog.cc:458] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:19.166198 7535 sys_catalog.cc:458] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b [sys.catalog]: This master's current role is: LEADER
W20260808 17:01:19.167518 7557 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:01:19.167578 7557 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:01:19.167639 7547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:19.167845 7547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:19.169337 7547 catalog_manager.cc:1347] Generated new cluster ID: e70b069fbb28471691044145295f9306
I20260808 17:01:19.169380 7547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:19.170352 7530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:19.159383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:33781 (local address 127.0.0.1:44659)
0808 17:01:19.159571 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:19.159581 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:19.159616 (+ 35us) server_negotiation.cc:407] Connection header received
0808 17:01:19.160228 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:19.160241 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:19.160334 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:19.160493 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:19.161182 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:19.162508 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:19.163263 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:19.163951 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:19.164069 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:19.166843 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:19.166847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:19.166847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:19.166861 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:19.168655 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:19.169141 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:19.169144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:19.169144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:19.169241 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:19.169498 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:19.169499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:19.169499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:19.169748 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:19.169917 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:19.170047 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 17:01:19.170151 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
I20260808 17:01:19.171332 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:19.179888 7530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:19.172858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:37913 (local address 127.0.0.1:44659)
0808 17:01:19.172892 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:19.172894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:19.172902 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:01:19.172922 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:19.172923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:19.172931 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:19.172994 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:19.173558 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:19.174276 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:19.175258 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:19.175574 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:19.175644 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:19.176961 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:19.176963 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:19.176963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:19.176970 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:19.178544 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:19.178956 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:19.178957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:19.178957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:19.179018 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:19.179258 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:19.179259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:19.179259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:19.179399 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:19.179507 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:19.179619 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 17:01:19.179690 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:01:19.185074 7547 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:19.185648 7547 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:19.189011 7547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d51b319219db4dd9b95941d8bc01572b: Generated new TSK 0
I20260808 17:01:19.189225 7547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:19.198132 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 7434
W20260808 17:01:19.209332 7415 connection.cc:535] client connection to 127.0.0.1:44659 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44659): Connection reset by peer (error 104)
2026-08-08T17:01:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1516 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:19 dist-test-slave-1nl6 krb5kdc[7573](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:19 dist-test-slave-1nl6 krb5kdc[7573](info): set up 2 sockets
Aug 08 17:01:19 dist-test-slave-1nl6 krb5kdc[7573](info): commencing operation
krb5kdc: starting...
W20260808 17:01:20.428061 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:20 dist-test-slave-1nl6 krb5kdc[7573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:01:20.547644 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34669
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:20.628342 7592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:20.628540 7592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:20.628562 7592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:20.629706 7592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:20.629766 7592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:20.629792 7592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:20.629828 7592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:20.629841 7592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:20.631384 7592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34669
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:20.631639 7592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:20.631825 7592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 17:01:20.636644 7592 server_base.cc:1034] running on GCE node
W20260808 17:01:20.636734 7598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:20.636673 7600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:20.636644 7597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:20.637099 7592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:20.637409 7592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:20.638558 7592 hybrid_clock.cc:648] HybridClock initialized: now 1786208480638543 us; error 37 us; skew 500 ppm
Aug 08 17:01:20 dist-test-slave-1nl6 krb5kdc[7573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208480, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:20.641013 7592 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:01:20.641444 7592 webserver.cc:460] Webserver started at http://127.0.0.1:42025/ using document root <none> and password file <none>
I20260808 17:01:20.641702 7592 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:20.641774 7592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:20.641927 7592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:20.643046 7592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "87560d4c8dbb46af812b2c79872e36b6"
format_stamp: "Formatted at 2026-08-08 17:01:20 on dist-test-slave-1nl6"
server_key: "0f7cfb283ce58a8520ca65206b33fb79"
server_key_iv: "d0ff9bcad7bfa71540a168b67b63ab1e"
server_key_version: "encryptionkey@0"
I20260808 17:01:20.643431 7592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "87560d4c8dbb46af812b2c79872e36b6"
format_stamp: "Formatted at 2026-08-08 17:01:20 on dist-test-slave-1nl6"
server_key: "0f7cfb283ce58a8520ca65206b33fb79"
server_key_iv: "d0ff9bcad7bfa71540a168b67b63ab1e"
server_key_version: "encryptionkey@0"
I20260808 17:01:20.644763 7592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:20.645494 7610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:20.645665 7592 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:20.645774 7592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "87560d4c8dbb46af812b2c79872e36b6"
format_stamp: "Formatted at 2026-08-08 17:01:20 on dist-test-slave-1nl6"
server_key: "0f7cfb283ce58a8520ca65206b33fb79"
server_key_iv: "d0ff9bcad7bfa71540a168b67b63ab1e"
server_key_version: "encryptionkey@0"
I20260808 17:01:20.645856 7592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:20.662047 7592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:20.664149 7592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:20.664327 7592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:20.668540 7592 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34669
I20260808 17:01:20.668581 7681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34669 every 8 connection(s)
I20260808 17:01:20.668975 7592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:20.669914 7682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:20.672318 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 7592
I20260808 17:01:20.672343 7682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6: Bootstrap starting.
I20260808 17:01:20.672453 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:20.672650 8249 external_mini_cluster.cc:1428] Setting key 2556d10216cfa0af0ae04f0a4119d153
I20260808 17:01:20.673087 7682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:20.673496 7682 log.cc:824] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:20.674278 7682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6: No bootstrap required, opened a new log
I20260808 17:01:20.676450 7682 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87560d4c8dbb46af812b2c79872e36b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } }
I20260808 17:01:20.676625 7682 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:20.676674 7682 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87560d4c8dbb46af812b2c79872e36b6, State: Initialized, Role: FOLLOWER
I20260808 17:01:20.676800 7682 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87560d4c8dbb46af812b2c79872e36b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } }
I20260808 17:01:20.676892 7682 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:20.676937 7682 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:20.676986 7682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:20.677729 7682 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87560d4c8dbb46af812b2c79872e36b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } }
I20260808 17:01:20.677830 7682 leader_election.cc:302] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87560d4c8dbb46af812b2c79872e36b6; no voters:
I20260808 17:01:20.677996 7682 leader_election.cc:288] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:20.678020 7689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:20.678211 7689 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [term 1 LEADER]: Becoming Leader. State: Replica: 87560d4c8dbb46af812b2c79872e36b6, State: Running, Role: LEADER
I20260808 17:01:20.678297 7689 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87560d4c8dbb46af812b2c79872e36b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } }
I20260808 17:01:20.678319 7682 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:20.678560 7686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:20.673189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:33943 (local address 127.0.0.1:34669)
0808 17:01:20.673383 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:20.673393 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:20.673426 (+ 33us) server_negotiation.cc:407] Connection header received
0808 17:01:20.673910 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:20.673918 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:20.673967 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:20.674085 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:20.674327 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:20.675307 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:20.676459 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:20.676958 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:20.677081 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:20.677691 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:20.677694 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:20.677695 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:20.677916 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:20.677959 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:20.677974 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:20.678080 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:20.678279 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 17:01:20.678310 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260808 17:01:20.678642 7689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87560d4c8dbb46af812b2c79872e36b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87560d4c8dbb46af812b2c79872e36b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } } }
I20260808 17:01:20.678740 7689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:20.678922 7689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87560d4c8dbb46af812b2c79872e36b6. Latest consensus state: current_term: 1 leader_uuid: "87560d4c8dbb46af812b2c79872e36b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87560d4c8dbb46af812b2c79872e36b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34669 } } }
I20260808 17:01:20.679064 7689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:20.679225 7700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:20.679471 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:20.679960 7700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:20.681674 7700 catalog_manager.cc:1347] Generated new cluster ID: 1b06e99e796c49c4907bd75de0a03817
I20260808 17:01:20.681731 7700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:20.684484 7686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:20.681039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:39421 (local address 127.0.0.1:34669)
0808 17:01:20.681073 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:20.681075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:20.681136 (+ 61us) server_negotiation.cc:407] Connection header received
0808 17:01:20.681223 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:20.681224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:20.681234 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:20.681293 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:20.681460 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:20.682103 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:20.683183 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:20.683492 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:20.683549 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:20.684221 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:20.684223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:20.684223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:20.684228 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:20.684241 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:20.684245 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:20.684299 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:20.684407 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 17:01:20.684416 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:01:20.702562 7700 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:20.703130 7700 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:20.708009 7700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87560d4c8dbb46af812b2c79872e36b6: Generated new TSK 0
I20260808 17:01:20.708184 7700 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:20.711949 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 7592
W20260808 17:01:20.727345 7570 connection.cc:535] client connection to 127.0.0.1:34669 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34669): Connection reset by peer (error 104)
2026-08-08T17:01:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1518 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:20 dist-test-slave-1nl6 krb5kdc[7726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:20 dist-test-slave-1nl6 krb5kdc[7726](info): set up 2 sockets
Aug 08 17:01:20 dist-test-slave-1nl6 krb5kdc[7726](info): commencing operation
krb5kdc: starting...
W20260808 17:01:21.936411 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:22 dist-test-slave-1nl6 krb5kdc[7726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 17:01:22.053447 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38691
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:22.134678 7740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:22.134939 7740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:22.134990 7740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:22.136144 7740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:22.136211 7740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:22.136238 7740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:22.136270 7740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:22.136296 7740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:22.137893 7740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38691
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:22.138163 7740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:22.138387 7740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:22.143289 7753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:22.143543 7750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:22.143270 7751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:22.143945 7740 server_base.cc:1034] running on GCE node
I20260808 17:01:22.144223 7740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:22.144572 7740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:22.145735 7740 hybrid_clock.cc:648] HybridClock initialized: now 1786208482145718 us; error 33 us; skew 500 ppm
Aug 08 17:01:22 dist-test-slave-1nl6 krb5kdc[7726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208482, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:22.148245 7740 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 17:01:22.148712 7740 webserver.cc:460] Webserver started at http://127.0.0.1:37195/ using document root <none> and password file <none>
I20260808 17:01:22.148943 7740 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:22.148988 7740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:22.149147 7740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:22.150213 7740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "8066df94697a40ea82f97f75540d8097"
format_stamp: "Formatted at 2026-08-08 17:01:22 on dist-test-slave-1nl6"
server_key: "fb557c398903bea70b4f457152ed3318"
server_key_iv: "c28f3081c1a52a6d36421f7a69706286"
server_key_version: "encryptionkey@0"
I20260808 17:01:22.150579 7740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "8066df94697a40ea82f97f75540d8097"
format_stamp: "Formatted at 2026-08-08 17:01:22 on dist-test-slave-1nl6"
server_key: "fb557c398903bea70b4f457152ed3318"
server_key_iv: "c28f3081c1a52a6d36421f7a69706286"
server_key_version: "encryptionkey@0"
I20260808 17:01:22.151975 7740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:01:22.152705 7762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:22.152902 7740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:01:22.152994 7740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "8066df94697a40ea82f97f75540d8097"
format_stamp: "Formatted at 2026-08-08 17:01:22 on dist-test-slave-1nl6"
server_key: "fb557c398903bea70b4f457152ed3318"
server_key_iv: "c28f3081c1a52a6d36421f7a69706286"
server_key_version: "encryptionkey@0"
I20260808 17:01:22.153049 7740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:22.156306 7740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:22.158329 7740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:22.158437 7740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:22.162307 7740 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38691
I20260808 17:01:22.162360 7838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38691 every 8 connection(s)
I20260808 17:01:22.162684 7740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:22.163632 7839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:22.165879 7839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097: Bootstrap starting.
I20260808 17:01:22.166476 7839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:22.166764 7839 log.cc:824] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:22.167492 7839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097: No bootstrap required, opened a new log
I20260808 17:01:22.167902 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 7740
I20260808 17:01:22.168037 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:22.168218 8249 external_mini_cluster.cc:1428] Setting key d17f5613a329948d21656f5b78c71932
I20260808 17:01:22.169534 7839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8066df94697a40ea82f97f75540d8097" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38691 } }
I20260808 17:01:22.169708 7839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:22.169751 7839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8066df94697a40ea82f97f75540d8097, State: Initialized, Role: FOLLOWER
I20260808 17:01:22.169891 7839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8066df94697a40ea82f97f75540d8097" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38691 } }
I20260808 17:01:22.170019 7839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:22.170087 7839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:22.170145 7839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:22.170850 7844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:22.168725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:60341 (local address 127.0.0.1:38691)
0808 17:01:22.168899 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:22.168906 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:22.168929 (+ 23us) server_negotiation.cc:407] Connection header received
0808 17:01:22.169515 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:22.169521 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:22.169545 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:22.169656 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:22.170378 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:22.170381 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:22.170382 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:22.170385 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:22.170422 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:22.170433 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:22.170489 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:22.170587 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 17:01:22.170620 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
I20260808 17:01:22.171286 7839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8066df94697a40ea82f97f75540d8097" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38691 } }
I20260808 17:01:22.171433 7839 leader_election.cc:302] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8066df94697a40ea82f97f75540d8097; no voters:
I20260808 17:01:22.171600 7839 leader_election.cc:288] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:22.171646 7845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:22.171792 7845 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [term 1 LEADER]: Becoming Leader. State: Replica: 8066df94697a40ea82f97f75540d8097, State: Running, Role: LEADER
I20260808 17:01:22.171844 7839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:22.171880 7845 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8066df94697a40ea82f97f75540d8097" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38691 } }
I20260808 17:01:22.171945 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:22.172194 7845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8066df94697a40ea82f97f75540d8097. Latest consensus state: current_term: 1 leader_uuid: "8066df94697a40ea82f97f75540d8097" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8066df94697a40ea82f97f75540d8097" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38691 } } }
I20260808 17:01:22.172212 7846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8066df94697a40ea82f97f75540d8097" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8066df94697a40ea82f97f75540d8097" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38691 } } }
I20260808 17:01:22.172276 7845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:22.172284 7846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:22.172513 7852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:22.173008 7852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:22.174726 7852 catalog_manager.cc:1347] Generated new cluster ID: 590c31880ba342e6ab5e429167d1e01c
I20260808 17:01:22.174785 7852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:22.175290 7844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:22.173663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.98:39265 (local address 127.0.0.1:38691)
0808 17:01:22.173699 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:22.173701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:22.174244 (+ 543us) server_negotiation.cc:407] Connection header received
0808 17:01:22.174352 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:22.174353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:22.174358 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:22.174406 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:22.175056 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:22.175057 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:22.175058 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:22.175062 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:22.175076 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:22.175080 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:22.175152 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:22.175211 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 17:01:22.175226 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:01:22.195533 7852 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:22.195709 7852 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:22.202430 7852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8066df94697a40ea82f97f75540d8097: Generated new TSK 0
I20260808 17:01:22.202623 7852 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:22.208215 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 7740
2026-08-08T17:01:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:22 dist-test-slave-1nl6 krb5kdc[7879](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:22 dist-test-slave-1nl6 krb5kdc[7879](info): set up 2 sockets
Aug 08 17:01:22 dist-test-slave-1nl6 krb5kdc[7879](info): commencing operation
krb5kdc: starting...
W20260808 17:01:23.420575 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:23 dist-test-slave-1nl6 krb5kdc[7879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208483, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 17:01:23.541520 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:33119
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:35947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:33119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:23.623270 7897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:23.623517 7897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:23.623556 7897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:23.624704 7897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:23.624774 7897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:23.624799 7897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:23.624820 7897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:23.624842 7897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:23.626300 7897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:35947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:33119
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:33119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:23.626605 7897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:23.626837 7897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:23.631783 7904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:23.631772 7907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:23.631768 7902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:23.632014 7897 server_base.cc:1034] running on GCE node
I20260808 17:01:23.632311 7897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:23.632613 7897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:23.633776 7897 hybrid_clock.cc:648] HybridClock initialized: now 1786208483633754 us; error 55 us; skew 500 ppm
Aug 08 17:01:23 dist-test-slave-1nl6 krb5kdc[7879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208483, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:23.640936 7897 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 17:01:23.641420 7897 webserver.cc:460] Webserver started at http://127.8.14.126:44903/ using document root <none> and password file <none>
I20260808 17:01:23.641654 7897 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:23.641722 7897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:23.641880 7897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:23.643005 7897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "1acf040d511b4954bf82d2c9cc30dca6"
format_stamp: "Formatted at 2026-08-08 17:01:23 on dist-test-slave-1nl6"
server_key: "a02a013f599405b2a43b8742f59fdb64"
server_key_iv: "ed317ec814366cf9dac449ff8303e6d9"
server_key_version: "encryptionkey@0"
I20260808 17:01:23.643446 7897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "1acf040d511b4954bf82d2c9cc30dca6"
format_stamp: "Formatted at 2026-08-08 17:01:23 on dist-test-slave-1nl6"
server_key: "a02a013f599405b2a43b8742f59fdb64"
server_key_iv: "ed317ec814366cf9dac449ff8303e6d9"
server_key_version: "encryptionkey@0"
I20260808 17:01:23.645236 7897 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260808 17:01:23.645996 7915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:23.646178 7897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:01:23.646281 7897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "1acf040d511b4954bf82d2c9cc30dca6"
format_stamp: "Formatted at 2026-08-08 17:01:23 on dist-test-slave-1nl6"
server_key: "a02a013f599405b2a43b8742f59fdb64"
server_key_iv: "ed317ec814366cf9dac449ff8303e6d9"
server_key_version: "encryptionkey@0"
I20260808 17:01:23.646371 7897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:23.672575 7897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:23.674499 7897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:23.674678 7897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:23.679219 7897 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:33119
I20260808 17:01:23.679255 7990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:33119 every 8 connection(s)
I20260808 17:01:23.679710 7897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:23.680605 7993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:23.683022 7993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6: Bootstrap starting.
I20260808 17:01:23.683640 7993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:23.683945 7993 log.cc:824] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:23.684649 7993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6: No bootstrap required, opened a new log
I20260808 17:01:23.686232 7993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1acf040d511b4954bf82d2c9cc30dca6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 33119 } }
I20260808 17:01:23.686375 7993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:23.686424 7993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1acf040d511b4954bf82d2c9cc30dca6, State: Initialized, Role: FOLLOWER
I20260808 17:01:23.686583 7993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1acf040d511b4954bf82d2c9cc30dca6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 33119 } }
I20260808 17:01:23.686654 7993 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:23.686695 7993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:23.686746 7993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:23.687464 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 7897
I20260808 17:01:23.687603 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:23.687826 8249 external_mini_cluster.cc:1428] Setting key 8a002b1573be2f988e11ad68dfb5f14e
I20260808 17:01:23.687903 7993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1acf040d511b4954bf82d2c9cc30dca6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 33119 } }
I20260808 17:01:23.688015 7993 leader_election.cc:302] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1acf040d511b4954bf82d2c9cc30dca6; no voters:
I20260808 17:01:23.688198 7993 leader_election.cc:288] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:23.688290 7999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:23.688673 7999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [term 1 LEADER]: Becoming Leader. State: Replica: 1acf040d511b4954bf82d2c9cc30dca6, State: Running, Role: LEADER
I20260808 17:01:23.688736 7993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:23.688853 7999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1acf040d511b4954bf82d2c9cc30dca6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 33119 } }
I20260808 17:01:23.689280 7999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1acf040d511b4954bf82d2c9cc30dca6. Latest consensus state: current_term: 1 leader_uuid: "1acf040d511b4954bf82d2c9cc30dca6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1acf040d511b4954bf82d2c9cc30dca6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 33119 } } }
I20260808 17:01:23.689430 7999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:23.689698 8014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:23.690056 8005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1acf040d511b4954bf82d2c9cc30dca6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1acf040d511b4954bf82d2c9cc30dca6" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 33119 } } }
I20260808 17:01:23.690169 8005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:23.690261 8014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:23.691881 8014 catalog_manager.cc:1347] Generated new cluster ID: 992d373a9aad4a98acea3c8c687c2fa8
I20260808 17:01:23.691941 8014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:01:23 dist-test-slave-1nl6 krb5kdc[7879](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208483, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:01:23.700253 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:23.688303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34617 (local address 127.8.14.126:33119)
0808 17:01:23.688543 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:23.688553 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:23.688593 (+ 40us) server_negotiation.cc:407] Connection header received
0808 17:01:23.688722 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:23.688731 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:23.688788 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:23.688987 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:23.689859 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:23.691219 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:23.692263 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:23.692847 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:23.692967 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:23.695861 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:23.695865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:23.695866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:23.695878 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:23.698333 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:23.698842 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:23.698845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:23.698845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:23.698988 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:23.699253 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:23.699253 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:23.699254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:23.699534 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:23.699682 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:23.699818 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 17:01:23.700035 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":117,"threads_started":1}
I20260808 17:01:23.701334 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:23.709830 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:23.702786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36231 (local address 127.8.14.126:33119)
0808 17:01:23.702847 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:23.702849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:23.702983 (+ 134us) server_negotiation.cc:407] Connection header received
0808 17:01:23.703084 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:23.703085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:23.703094 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:23.703120 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:23.703719 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:23.704465 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:23.705422 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:23.705725 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:23.705760 (+ 35us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:23.707148 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:23.707149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:23.707150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:23.707156 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:23.708616 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:23.709025 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:23.709026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:23.709026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:23.709074 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:23.709273 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:23.709274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:23.709274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:23.709381 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:23.709466 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:23.709685 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 17:01:23.709745 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 17:01:23.710814 8014 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:23.711306 8014 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:23.717295 8014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1acf040d511b4954bf82d2c9cc30dca6: Generated new TSK 0
I20260808 17:01:23.717465 8014 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:23.725144 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 7897
W20260808 17:01:23.735877 7876 connection.cc:535] client connection to 127.8.14.126:33119 recv error: Network error: failed to read from TLS socket (remote: 127.8.14.126:33119): Connection reset by peer (error 104)
2026-08-08T17:01:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1516 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:23 dist-test-slave-1nl6 krb5kdc[8035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:23 dist-test-slave-1nl6 krb5kdc[8035](info): set up 2 sockets
Aug 08 17:01:23 dist-test-slave-1nl6 krb5kdc[8035](info): commencing operation
krb5kdc: starting...
W20260808 17:01:24.941174 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:25 dist-test-slave-1nl6 krb5kdc[8035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 17:01:25.062052 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:42155
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:38557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:42155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:25.143517 8053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:25.143746 8053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:25.143784 8053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:25.144896 8053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:25.144973 8053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:25.144996 8053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:25.145018 8053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:25.145048 8053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:25.146515 8053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:38557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:42155
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:42155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:25.146799 8053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:25.147064 8053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:25.151933 8061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:25.151929 8058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:25.151925 8059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:25.151962 8053 server_base.cc:1034] running on GCE node
I20260808 17:01:25.152438 8053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:25.152734 8053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:25.153908 8053 hybrid_clock.cc:648] HybridClock initialized: now 1786208485153894 us; error 44 us; skew 500 ppm
Aug 08 17:01:25 dist-test-slave-1nl6 krb5kdc[8035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208485, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:25.156523 8053 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 17:01:25.156966 8053 webserver.cc:460] Webserver started at http://127.8.14.126:44903/ using document root <none> and password file <none>
I20260808 17:01:25.157212 8053 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:25.157258 8053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:25.157423 8053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:25.158421 8053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "97e0835027c544a8b265a6d9d00c870d"
format_stamp: "Formatted at 2026-08-08 17:01:25 on dist-test-slave-1nl6"
server_key: "f3348cc33d9064956f54c4e887097f83"
server_key_iv: "a257a03301339f107ffcb38c7579604e"
server_key_version: "encryptionkey@0"
I20260808 17:01:25.158777 8053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "97e0835027c544a8b265a6d9d00c870d"
format_stamp: "Formatted at 2026-08-08 17:01:25 on dist-test-slave-1nl6"
server_key: "f3348cc33d9064956f54c4e887097f83"
server_key_iv: "a257a03301339f107ffcb38c7579604e"
server_key_version: "encryptionkey@0"
I20260808 17:01:25.160140 8053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:01:25.160866 8072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:25.161054 8053 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:01:25.161123 8053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "97e0835027c544a8b265a6d9d00c870d"
format_stamp: "Formatted at 2026-08-08 17:01:25 on dist-test-slave-1nl6"
server_key: "f3348cc33d9064956f54c4e887097f83"
server_key_iv: "a257a03301339f107ffcb38c7579604e"
server_key_version: "encryptionkey@0"
I20260808 17:01:25.161211 8053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:25.175895 8053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:25.178035 8053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:25.178189 8053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:25.182189 8053 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:42155
I20260808 17:01:25.182260 8151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:42155 every 8 connection(s)
I20260808 17:01:25.182569 8053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:25.183458 8152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:25.185640 8152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d: Bootstrap starting.
I20260808 17:01:25.186192 8152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:25.186434 8152 log.cc:824] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:25.187107 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 8053
I20260808 17:01:25.187214 8152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d: No bootstrap required, opened a new log
I20260808 17:01:25.187230 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:25.187444 8249 external_mini_cluster.cc:1428] Setting key d91ea6e917ba4ebf457eeec2ad2355a9
I20260808 17:01:25.189913 8152 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e0835027c544a8b265a6d9d00c870d" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42155 } }
I20260808 17:01:25.190124 8152 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:25.190164 8152 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e0835027c544a8b265a6d9d00c870d, State: Initialized, Role: FOLLOWER
I20260808 17:01:25.190303 8152 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e0835027c544a8b265a6d9d00c870d" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42155 } }
I20260808 17:01:25.190385 8152 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:25.190415 8152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:25.190461 8152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:25.191162 8152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e0835027c544a8b265a6d9d00c870d" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42155 } }
I20260808 17:01:25.191255 8152 leader_election.cc:302] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97e0835027c544a8b265a6d9d00c870d; no voters:
I20260808 17:01:25.191484 8152 leader_election.cc:288] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:25.191514 8158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:25.191629 8158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [term 1 LEADER]: Becoming Leader. State: Replica: 97e0835027c544a8b265a6d9d00c870d, State: Running, Role: LEADER
I20260808 17:01:25.191702 8158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e0835027c544a8b265a6d9d00c870d" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42155 } }
I20260808 17:01:25.191764 8152 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:25.192024 8158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97e0835027c544a8b265a6d9d00c870d. Latest consensus state: current_term: 1 leader_uuid: "97e0835027c544a8b265a6d9d00c870d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e0835027c544a8b265a6d9d00c870d" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42155 } } }
I20260808 17:01:25.192188 8158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:25.192385 8158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97e0835027c544a8b265a6d9d00c870d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97e0835027c544a8b265a6d9d00c870d" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42155 } } }
I20260808 17:01:25.192513 8158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:25.192911 8165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:25.193068 8157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:25.187951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41535 (local address 127.8.14.126:42155)
0808 17:01:25.188160 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:25.188167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:25.188191 (+ 24us) server_negotiation.cc:407] Connection header received
0808 17:01:25.188622 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:25.188627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:25.188672 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:25.188778 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:25.189009 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:25.189877 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:25.190999 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:25.191491 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:25.191606 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:25.192324 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:25.192328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:25.192329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:25.192527 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:25.192562 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:25.192577 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:25.192663 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:25.192750 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 17:01:25.192776 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":112,"threads_started":1}
I20260808 17:01:25.193449 8165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:25.194185 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:25.195500 8165 catalog_manager.cc:1347] Generated new cluster ID: db799a01c38742ce99751a6eaf43c5ab
I20260808 17:01:25.195559 8165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:25.205782 8157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:25.195820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55233 (local address 127.8.14.126:42155)
0808 17:01:25.195857 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:25.195859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:25.203035 (+ 7176us) server_negotiation.cc:407] Connection header received
0808 17:01:25.203101 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:25.203102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:25.203111 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:25.203187 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:25.203370 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:25.203820 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:25.204552 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:25.204756 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:25.204830 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:25.205509 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:25.205510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:25.205511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:25.205515 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:25.205528 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:25.205532 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:25.205604 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:25.205676 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 17:01:25.205685 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 17:01:25.207784 8165 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:25.208279 8165 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:25.213829 8165 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97e0835027c544a8b265a6d9d00c870d: Generated new TSK 0
I20260808 17:01:25.214017 8165 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:25.219772 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 8053
W20260808 17:01:25.240782 8033 connection.cc:535] client connection to 127.8.14.126:42155 recv error: Network error: failed to read from TLS socket (remote: 127.8.14.126:42155): Connection reset by peer (error 104)
2026-08-08T17:01:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1504 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:25 dist-test-slave-1nl6 krb5kdc[8187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:25 dist-test-slave-1nl6 krb5kdc[8187](info): set up 2 sockets
Aug 08 17:01:25 dist-test-slave-1nl6 krb5kdc[8187](info): commencing operation
krb5kdc: starting...
W20260808 17:01:26.451185 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:26 dist-test-slave-1nl6 krb5kdc[8187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 17:01:26.569729 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:41737
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:45553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:41737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:26.651072 8206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:26.651273 8206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:26.651296 8206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:26.652395 8206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:26.652434 8206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:26.652448 8206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:26.652460 8206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:26.652472 8206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:26.653929 8206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:45553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:41737
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:41737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:26.654124 8206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:26.654284 8206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:26.659145 8215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:26.659143 8217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:26.659145 8214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:26.659411 8206 server_base.cc:1034] running on GCE node
I20260808 17:01:26.659665 8206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:26.659974 8206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:26.661121 8206 hybrid_clock.cc:648] HybridClock initialized: now 1786208486661109 us; error 28 us; skew 500 ppm
Aug 08 17:01:26 dist-test-slave-1nl6 krb5kdc[8187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208486, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:26.663704 8206 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 17:01:26.664181 8206 webserver.cc:460] Webserver started at http://127.8.14.126:38043/ using document root <none> and password file <none>
I20260808 17:01:26.664404 8206 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:26.664449 8206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:26.664615 8206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:26.665652 8206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "479cd52f726248ea9718fa622737c712"
format_stamp: "Formatted at 2026-08-08 17:01:26 on dist-test-slave-1nl6"
server_key: "0e3395c67629777356fba90ef57e5e4b"
server_key_iv: "3ee30cff5752d0993ab6f4c3f570ebb3"
server_key_version: "encryptionkey@0"
I20260808 17:01:26.666028 8206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "479cd52f726248ea9718fa622737c712"
format_stamp: "Formatted at 2026-08-08 17:01:26 on dist-test-slave-1nl6"
server_key: "0e3395c67629777356fba90ef57e5e4b"
server_key_iv: "3ee30cff5752d0993ab6f4c3f570ebb3"
server_key_version: "encryptionkey@0"
I20260808 17:01:26.667421 8206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:01:26.668236 8226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:26.668439 8206 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:01:26.668536 8206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "479cd52f726248ea9718fa622737c712"
format_stamp: "Formatted at 2026-08-08 17:01:26 on dist-test-slave-1nl6"
server_key: "0e3395c67629777356fba90ef57e5e4b"
server_key_iv: "3ee30cff5752d0993ab6f4c3f570ebb3"
server_key_version: "encryptionkey@0"
I20260808 17:01:26.668612 8206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:26.671943 8206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:26.674229 8206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:26.674389 8206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:26.678484 8206 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:41737
I20260808 17:01:26.678514 8307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:41737 every 8 connection(s)
I20260808 17:01:26.678838 8206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:26.679785 8308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:26.681952 8308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712: Bootstrap starting.
I20260808 17:01:26.682464 8308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:26.682708 8308 log.cc:824] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:26.683411 8308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712: No bootstrap required, opened a new log
I20260808 17:01:26.684306 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 8206
I20260808 17:01:26.684412 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:26.684638 8249 external_mini_cluster.cc:1428] Setting key 2419bfec5c035d597cd18324df547461
I20260808 17:01:26.685575 8308 raft_consensus.cc:348] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479cd52f726248ea9718fa622737c712" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41737 } }
I20260808 17:01:26.685761 8308 raft_consensus.cc:374] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:26.685804 8308 raft_consensus.cc:729] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 479cd52f726248ea9718fa622737c712, State: Initialized, Role: FOLLOWER
I20260808 17:01:26.686009 8308 consensus_queue.cc:260] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479cd52f726248ea9718fa622737c712" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41737 } }
I20260808 17:01:26.686131 8308 raft_consensus.cc:388] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:26.686192 8308 raft_consensus.cc:482] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:26.686254 8308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:26.686791 8314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:26.685131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51041 (local address 127.8.14.126:41737)
0808 17:01:26.685323 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:26.685330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:26.685355 (+ 25us) server_negotiation.cc:407] Connection header received
0808 17:01:26.685536 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:26.685541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:26.685572 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:26.685677 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:26.686332 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:26.686336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:26.686337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:26.686341 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:26.686384 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:26.686395 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:26.686450 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:26.686540 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 17:01:26.686571 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260808 17:01:26.687327 8308 raft_consensus.cc:504] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479cd52f726248ea9718fa622737c712" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41737 } }
I20260808 17:01:26.687448 8308 leader_election.cc:302] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 479cd52f726248ea9718fa622737c712; no voters:
I20260808 17:01:26.687638 8308 leader_election.cc:288] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:26.687680 8315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:26.687747 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:26.687829 8315 raft_consensus.cc:686] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [term 1 LEADER]: Becoming Leader. State: Replica: 479cd52f726248ea9718fa622737c712, State: Running, Role: LEADER
I20260808 17:01:26.687889 8308 sys_catalog.cc:564] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:26.687912 8315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479cd52f726248ea9718fa622737c712" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41737 } }
I20260808 17:01:26.688243 8315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "479cd52f726248ea9718fa622737c712" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479cd52f726248ea9718fa622737c712" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41737 } } }
I20260808 17:01:26.688333 8315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:26.688531 8315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 479cd52f726248ea9718fa622737c712. Latest consensus state: current_term: 1 leader_uuid: "479cd52f726248ea9718fa622737c712" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "479cd52f726248ea9718fa622737c712" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41737 } } }
I20260808 17:01:26.688673 8315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:26.689057 8326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:26.689777 8326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:26.690600 8314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:26.689329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46367 (local address 127.8.14.126:41737)
0808 17:01:26.689363 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:26.689365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:26.689632 (+ 267us) server_negotiation.cc:407] Connection header received
0808 17:01:26.689709 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:26.689710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:26.689716 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:26.689767 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:26.690379 (+ 612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:26.690380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:26.690380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:26.690384 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:26.690399 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:26.690404 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:26.690461 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:26.690525 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 17:01:26.690538 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:01:26.691449 8326 catalog_manager.cc:1347] Generated new cluster ID: ee56fca8b07a4899ae46fcb79442e5e5
I20260808 17:01:26.691502 8326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:26.717115 8326 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:26.717290 8326 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:26.726648 8326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 479cd52f726248ea9718fa622737c712: Generated new TSK 0
I20260808 17:01:26.726830 8326 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:26.733685 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 8206
2026-08-08T17:01:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1504 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:26 dist-test-slave-1nl6 krb5kdc[8355](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:26 dist-test-slave-1nl6 krb5kdc[8355](info): set up 2 sockets
Aug 08 17:01:26 dist-test-slave-1nl6 krb5kdc[8355](info): commencing operation
krb5kdc: starting...
W20260808 17:01:27.947453 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:28 dist-test-slave-1nl6 krb5kdc[8355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 17:01:28.065418 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:41201
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:40073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:41201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:28.152597 8372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:28.152869 8372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:28.152904 8372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:28.154480 8372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:28.154537 8372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:28.154556 8372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:28.154573 8372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:28.154589 8372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:28.156687 8372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:40073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:41201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:41201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:28.156952 8372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:28.157205 8372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:28.162096 8378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:28.162058 8377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:28.162284 8372 server_base.cc:1034] running on GCE node
W20260808 17:01:28.162050 8382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:28.162590 8372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:28.162878 8372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:28.164055 8372 hybrid_clock.cc:648] HybridClock initialized: now 1786208488164029 us; error 44 us; skew 500 ppm
Aug 08 17:01:28 dist-test-slave-1nl6 krb5kdc[8355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208488, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:28.166514 8372 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 17:01:28.166989 8372 webserver.cc:460] Webserver started at http://127.8.14.126:41521/ using document root <none> and password file <none>
I20260808 17:01:28.167253 8372 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:28.167325 8372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:28.167466 8372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:28.168534 8372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "6573895d16244600a1588a3b74b80362"
format_stamp: "Formatted at 2026-08-08 17:01:28 on dist-test-slave-1nl6"
server_key: "f0fb37fb55775377c8655b4a3079ed06"
server_key_iv: "84e6cfc0a9978fce1c2cb3ec12d24b89"
server_key_version: "encryptionkey@0"
I20260808 17:01:28.168911 8372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "6573895d16244600a1588a3b74b80362"
format_stamp: "Formatted at 2026-08-08 17:01:28 on dist-test-slave-1nl6"
server_key: "f0fb37fb55775377c8655b4a3079ed06"
server_key_iv: "84e6cfc0a9978fce1c2cb3ec12d24b89"
server_key_version: "encryptionkey@0"
I20260808 17:01:28.170257 8372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:01:28.171065 8392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:28.171242 8372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:01:28.171358 8372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "6573895d16244600a1588a3b74b80362"
format_stamp: "Formatted at 2026-08-08 17:01:28 on dist-test-slave-1nl6"
server_key: "f0fb37fb55775377c8655b4a3079ed06"
server_key_iv: "84e6cfc0a9978fce1c2cb3ec12d24b89"
server_key_version: "encryptionkey@0"
I20260808 17:01:28.171433 8372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:28.187719 8372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:28.189496 8372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:28.189657 8372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:28.193647 8372 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:41201
I20260808 17:01:28.193696 8460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:41201 every 8 connection(s)
I20260808 17:01:28.194056 8372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:28.194976 8462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:28.197184 8462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362: Bootstrap starting.
I20260808 17:01:28.197782 8462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:28.198071 8462 log.cc:824] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:28.198776 8462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362: No bootstrap required, opened a new log
I20260808 17:01:28.200395 8462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6573895d16244600a1588a3b74b80362" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41201 } }
I20260808 17:01:28.200531 8462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:28.200600 8462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6573895d16244600a1588a3b74b80362, State: Initialized, Role: FOLLOWER
I20260808 17:01:28.200771 8462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6573895d16244600a1588a3b74b80362" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41201 } }
I20260808 17:01:28.200860 8462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:28.200867 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 8372
I20260808 17:01:28.200884 8462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:28.200949 8462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:28.200979 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:28.201150 8249 external_mini_cluster.cc:1428] Setting key dad11dd17f5d795de24f71601a53c72c
I20260808 17:01:28.201984 8462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6573895d16244600a1588a3b74b80362" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41201 } }
I20260808 17:01:28.202075 8462 leader_election.cc:302] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6573895d16244600a1588a3b74b80362; no voters:
I20260808 17:01:28.202273 8462 leader_election.cc:288] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:28.202302 8470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:28.202410 8470 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [term 1 LEADER]: Becoming Leader. State: Replica: 6573895d16244600a1588a3b74b80362, State: Running, Role: LEADER
I20260808 17:01:28.202538 8462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:28.202555 8470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6573895d16244600a1588a3b74b80362" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41201 } }
I20260808 17:01:28.203073 8472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6573895d16244600a1588a3b74b80362. Latest consensus state: current_term: 1 leader_uuid: "6573895d16244600a1588a3b74b80362" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6573895d16244600a1588a3b74b80362" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41201 } } }
I20260808 17:01:28.203182 8472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:28.203095 8471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6573895d16244600a1588a3b74b80362" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6573895d16244600a1588a3b74b80362" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 41201 } } }
I20260808 17:01:28.203408 8471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:28.204197 8483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:28.204667 8483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:28.206007 8483 catalog_manager.cc:1347] Generated new cluster ID: df97a1e3e6144035bd16a7f9badba15f
I20260808 17:01:28.206058 8483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 17:01:28 dist-test-slave-1nl6 krb5kdc[8355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:01:28.213500 8467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:28.201653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35753 (local address 127.8.14.126:41201)
0808 17:01:28.201878 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:28.201888 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:28.201923 (+ 35us) server_negotiation.cc:407] Connection header received
0808 17:01:28.202015 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:28.202020 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:28.202089 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:28.202217 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:28.203172 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:28.204594 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:28.205619 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:28.206184 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:28.206297 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:28.209277 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:28.209281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:28.209281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:28.209293 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:28.211397 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:28.211816 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:28.211819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:28.211820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:28.211938 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:28.212166 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:28.212167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:28.212167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:28.212469 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:28.212645 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:28.213159 (+ 514us) server_negotiation.cc:299] Negotiation successful
0808 17:01:28.213297 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":114,"threads_started":1}
I20260808 17:01:28.214336 8483 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:28.214478 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:28.215057 8483 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:28.222940 8467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:28.215990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55505 (local address 127.8.14.126:41201)
0808 17:01:28.216056 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:28.216058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:28.216068 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:01:28.216143 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:28.216144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:28.216153 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:28.216204 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:28.216808 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:28.217524 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:28.218525 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:28.218841 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:28.218954 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:28.220169 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:28.220171 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:28.220171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:28.220178 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:28.221819 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:28.222160 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:28.222161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:28.222161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:28.222217 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:28.222416 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:28.222416 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:28.222417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:28.222533 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:28.222625 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:28.222743 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 17:01:28.222822 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 17:01:28.226768 8483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6573895d16244600a1588a3b74b80362: Generated new TSK 0
I20260808 17:01:28.226963 8483 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:28.232173 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 8372
2026-08-08T17:01:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:28 dist-test-slave-1nl6 krb5kdc[8512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:28 dist-test-slave-1nl6 krb5kdc[8512](info): set up 2 sockets
Aug 08 17:01:28 dist-test-slave-1nl6 krb5kdc[8512](info): commencing operation
krb5kdc: starting...
W20260808 17:01:29.451887 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:29 dist-test-slave-1nl6 krb5kdc[8512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 17:01:29.571485 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:34881
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:41745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:34881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:29.653429 8530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:29.653646 8530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:29.653667 8530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:29.654755 8530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:29.654796 8530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:29.654809 8530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:29.654861 8530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:29.654887 8530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:29.656353 8530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:41745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:34881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:34881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:29.656586 8530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:29.656821 8530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:29.661688 8541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:29.661646 8539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:29.661715 8530 server_base.cc:1034] running on GCE node
W20260808 17:01:29.661653 8536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:29.662092 8530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:29.662412 8530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:29.663566 8530 hybrid_clock.cc:648] HybridClock initialized: now 1786208489663557 us; error 38 us; skew 500 ppm
Aug 08 17:01:29 dist-test-slave-1nl6 krb5kdc[8512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208489, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:29.670408 8530 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 17:01:29.670853 8530 webserver.cc:460] Webserver started at http://127.8.14.126:45643/ using document root <none> and password file <none>
I20260808 17:01:29.671095 8530 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:29.671165 8530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:29.671324 8530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:29.672405 8530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "c6451c6bbd5b492e889d5ec009bcd77e"
format_stamp: "Formatted at 2026-08-08 17:01:29 on dist-test-slave-1nl6"
server_key: "6d0455fb645ac9581d699f66194fee60"
server_key_iv: "9d74ae89de12a31f3a75c868be65cdca"
server_key_version: "encryptionkey@0"
I20260808 17:01:29.672780 8530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "c6451c6bbd5b492e889d5ec009bcd77e"
format_stamp: "Formatted at 2026-08-08 17:01:29 on dist-test-slave-1nl6"
server_key: "6d0455fb645ac9581d699f66194fee60"
server_key_iv: "9d74ae89de12a31f3a75c868be65cdca"
server_key_version: "encryptionkey@0"
I20260808 17:01:29.674172 8530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:01:29.674876 8551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:29.675077 8530 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:29.675175 8530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "c6451c6bbd5b492e889d5ec009bcd77e"
format_stamp: "Formatted at 2026-08-08 17:01:29 on dist-test-slave-1nl6"
server_key: "6d0455fb645ac9581d699f66194fee60"
server_key_iv: "9d74ae89de12a31f3a75c868be65cdca"
server_key_version: "encryptionkey@0"
I20260808 17:01:29.675261 8530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:29.699934 8530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:29.701727 8530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:29.701898 8530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:29.706166 8530 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:34881
I20260808 17:01:29.706246 8625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:34881 every 8 connection(s)
I20260808 17:01:29.706599 8530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:29.707593 8626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:29.709923 8626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e: Bootstrap starting.
I20260808 17:01:29.710520 8626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:29.710824 8626 log.cc:824] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:29.711546 8626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e: No bootstrap required, opened a new log
I20260808 17:01:29.713166 8626 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6451c6bbd5b492e889d5ec009bcd77e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34881 } }
I20260808 17:01:29.713313 8626 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:29.713335 8626 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6451c6bbd5b492e889d5ec009bcd77e, State: Initialized, Role: FOLLOWER
I20260808 17:01:29.713521 8626 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6451c6bbd5b492e889d5ec009bcd77e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34881 } }
I20260808 17:01:29.713608 8626 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:29.713665 8626 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:29.713718 8626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:29.714596 8626 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6451c6bbd5b492e889d5ec009bcd77e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34881 } }
I20260808 17:01:29.714718 8626 leader_election.cc:302] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6451c6bbd5b492e889d5ec009bcd77e; no voters:
I20260808 17:01:29.714941 8626 leader_election.cc:288] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:29.714984 8631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:29.715157 8631 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [term 1 LEADER]: Becoming Leader. State: Replica: c6451c6bbd5b492e889d5ec009bcd77e, State: Running, Role: LEADER
I20260808 17:01:29.715217 8626 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:29.715272 8631 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6451c6bbd5b492e889d5ec009bcd77e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34881 } }
I20260808 17:01:29.715704 8633 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6451c6bbd5b492e889d5ec009bcd77e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6451c6bbd5b492e889d5ec009bcd77e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34881 } } }
I20260808 17:01:29.715806 8633 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:29.716252 8644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:29.716277 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 8530
I20260808 17:01:29.716409 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:29.715945 8634 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6451c6bbd5b492e889d5ec009bcd77e. Latest consensus state: current_term: 1 leader_uuid: "c6451c6bbd5b492e889d5ec009bcd77e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6451c6bbd5b492e889d5ec009bcd77e" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34881 } } }
I20260808 17:01:29.716650 8634 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:29.716665 8249 external_mini_cluster.cc:1428] Setting key 472e7fd14e70e3723743b54c3365c44a
I20260808 17:01:29.716859 8644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:29.718760 8644 catalog_manager.cc:1347] Generated new cluster ID: 15bebfa880e44e788b05b07818814e48
I20260808 17:01:29.718819 8644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:29.723112 8658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:29.717344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58817 (local address 127.8.14.126:34881)
0808 17:01:29.717557 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:29.717566 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:29.717816 (+ 250us) server_negotiation.cc:407] Connection header received
0808 17:01:29.717935 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:29.717943 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:29.717997 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:29.718174 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:29.718443 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:29.719860 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:29.720942 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:29.721579 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:29.721708 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:29.722372 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:29.722377 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:29.722377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:29.722608 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:29.722657 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:29.722671 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:29.722743 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:29.722810 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 17:01:29.722873 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260808 17:01:29.724007 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:29.730037 8644 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:29.730782 8644 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:29.730872 8658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:29.727243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51801 (local address 127.8.14.126:34881)
0808 17:01:29.727277 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:29.727279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:29.727400 (+ 121us) server_negotiation.cc:407] Connection header received
0808 17:01:29.727429 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:29.727430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:29.727439 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:29.727504 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:29.727645 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:29.728392 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:29.729379 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:29.729685 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:29.729749 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:29.730451 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:29.730452 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:29.730452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:29.730457 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:29.730471 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:29.730475 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:29.730523 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:29.730582 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 17:01:29.730755 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":147,"server-negotiator.queue_time_us":18}
I20260808 17:01:29.736493 8644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6451c6bbd5b492e889d5ec009bcd77e: Generated new TSK 0
I20260808 17:01:29.736663 8644 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:29.738523 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 8530
2026-08-08T17:01:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1505 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:29 dist-test-slave-1nl6 krb5kdc[8672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:29 dist-test-slave-1nl6 krb5kdc[8672](info): set up 2 sockets
Aug 08 17:01:29 dist-test-slave-1nl6 krb5kdc[8672](info): commencing operation
krb5kdc: starting...
W20260808 17:01:30.970536 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:31 dist-test-slave-1nl6 krb5kdc[8672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 17:01:31.087697 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:42565
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:46739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:42565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:31.167827 8694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:31.168013 8694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:31.168036 8694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:31.169082 8694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:31.169117 8694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:31.169131 8694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:31.169143 8694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:31.169155 8694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:31.170522 8694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:46739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:42565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:42565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:31.170702 8694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:31.170871 8694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:31.175552 8702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:31.175595 8701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:31.175570 8704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:31.175829 8694 server_base.cc:1034] running on GCE node
I20260808 17:01:31.176049 8694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:31.176321 8694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:31.177460 8694 hybrid_clock.cc:648] HybridClock initialized: now 1786208491177440 us; error 37 us; skew 500 ppm
Aug 08 17:01:31 dist-test-slave-1nl6 krb5kdc[8672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208491, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:31.179852 8694 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 17:01:31.180217 8694 webserver.cc:460] Webserver started at http://127.8.14.126:35773/ using document root <none> and password file <none>
I20260808 17:01:31.180418 8694 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:31.180459 8694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:31.180547 8694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:31.181562 8694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "38c720bdbdb64a7dac325b7be339687f"
format_stamp: "Formatted at 2026-08-08 17:01:31 on dist-test-slave-1nl6"
server_key: "571ab941f96762f9341147d3f4fb5862"
server_key_iv: "724b0f4f794e757d078e0f40abd9ed13"
server_key_version: "encryptionkey@0"
I20260808 17:01:31.181861 8694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "38c720bdbdb64a7dac325b7be339687f"
format_stamp: "Formatted at 2026-08-08 17:01:31 on dist-test-slave-1nl6"
server_key: "571ab941f96762f9341147d3f4fb5862"
server_key_iv: "724b0f4f794e757d078e0f40abd9ed13"
server_key_version: "encryptionkey@0"
I20260808 17:01:31.183143 8694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:01:31.183844 8713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:31.184065 8694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:31.184149 8694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "38c720bdbdb64a7dac325b7be339687f"
format_stamp: "Formatted at 2026-08-08 17:01:31 on dist-test-slave-1nl6"
server_key: "571ab941f96762f9341147d3f4fb5862"
server_key_iv: "724b0f4f794e757d078e0f40abd9ed13"
server_key_version: "encryptionkey@0"
I20260808 17:01:31.184203 8694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:31.187326 8694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:31.189028 8694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:31.189127 8694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:31.192857 8694 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:42565
I20260808 17:01:31.192929 8778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:42565 every 8 connection(s)
I20260808 17:01:31.193217 8694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:31.194187 8779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:31.196333 8779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f: Bootstrap starting.
I20260808 17:01:31.196833 8779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:31.197064 8779 log.cc:824] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:31.197712 8779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f: No bootstrap required, opened a new log
I20260808 17:01:31.199332 8779 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c720bdbdb64a7dac325b7be339687f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42565 } }
I20260808 17:01:31.199471 8779 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 17:01:31.199493 8779 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c720bdbdb64a7dac325b7be339687f, State: Initialized, Role: FOLLOWER
I20260808 17:01:31.199610 8779 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c720bdbdb64a7dac325b7be339687f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42565 } }
I20260808 17:01:31.199687 8779 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 17:01:31.199708 8779 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 17:01:31.199735 8779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:31.200452 8779 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c720bdbdb64a7dac325b7be339687f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42565 } }
I20260808 17:01:31.200539 8779 leader_election.cc:302] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38c720bdbdb64a7dac325b7be339687f; no voters:
I20260808 17:01:31.200659 8779 leader_election.cc:288] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 17:01:31.200722 8783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:31.200893 8783 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [term 1 LEADER]: Becoming Leader. State: Replica: 38c720bdbdb64a7dac325b7be339687f, State: Running, Role: LEADER
I20260808 17:01:31.200930 8779 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:31.200981 8783 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c720bdbdb64a7dac325b7be339687f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42565 } }
I20260808 17:01:31.201354 8785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38c720bdbdb64a7dac325b7be339687f. Latest consensus state: current_term: 1 leader_uuid: "38c720bdbdb64a7dac325b7be339687f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c720bdbdb64a7dac325b7be339687f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42565 } } }
I20260808 17:01:31.201327 8784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38c720bdbdb64a7dac325b7be339687f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38c720bdbdb64a7dac325b7be339687f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 42565 } } }
I20260808 17:01:31.201431 8784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:31.201427 8785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:31.201699 8794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:31.201833 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 8694
I20260808 17:01:31.201947 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:31.202147 8249 external_mini_cluster.cc:1428] Setting key 7d30936bd34d48d31e3b6df9ded17248
I20260808 17:01:31.202445 8794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:31.204213 8794 catalog_manager.cc:1347] Generated new cluster ID: 8ac4787da1e041a8aeefe2fb2e62fa88
I20260808 17:01:31.204274 8794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:31.204519 8805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:31.202610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47779 (local address 127.8.14.126:42565)
0808 17:01:31.202998 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:31.203007 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:31.203042 (+ 35us) server_negotiation.cc:407] Connection header received
0808 17:01:31.203153 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:31.203160 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:31.203189 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:31.203329 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:31.203979 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:31.203983 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:31.203983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:31.203988 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:31.204032 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:31.204046 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:31.204273 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:31.204316 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 17:01:31.204357 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":174,"threads_started":1}
I20260808 17:01:31.205435 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:31.208045 8805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:31.206970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55783 (local address 127.8.14.126:42565)
0808 17:01:31.207001 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:31.207003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:31.207147 (+ 144us) server_negotiation.cc:407] Connection header received
0808 17:01:31.207220 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:31.207221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:31.207225 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:31.207275 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:31.207815 (+ 540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:31.207815 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:31.207816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 17:01:31.207819 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:31.207832 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 17:01:31.207836 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:31.207890 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:31.207947 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 17:01:31.207959 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 17:01:31.216156 8794 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:31.216300 8794 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:31.222098 8794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38c720bdbdb64a7dac325b7be339687f: Generated new TSK 0
I20260808 17:01:31.222266 8794 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:31.231465 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 8694
2026-08-08T17:01:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1490 ms)
[----------] 18 tests from AuthTokenIssuingTest (27016 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:31 dist-test-slave-1nl6 krb5kdc[8820](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:31 dist-test-slave-1nl6 krb5kdc[8820](info): set up 2 sockets
Aug 08 17:01:31 dist-test-slave-1nl6 krb5kdc[8820](info): commencing operation
krb5kdc: starting...
W20260808 17:01:32.456248 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:32 dist-test-slave-1nl6 krb5kdc[8820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 17:01:32.587280 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:37083
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:35489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:37083,127.8.14.125:42321,127.8.14.124:46793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:32.669414 8840 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:01:32.669623 8840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:32.669641 8840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:32.669689 8840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:32.670809 8840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:32.670849 8840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:32.670863 8840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:32.670938 8840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:32.670964 8840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:32.672462 8840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:35489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:37083,127.8.14.125:42321,127.8.14.124:46793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:37083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:32.672714 8840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:32.672947 8840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:32.677971 8858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:32.678013 8855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:32.678045 8853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:32.678134 8840 server_base.cc:1034] running on GCE node
I20260808 17:01:32.678339 8840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:32.678570 8840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:32.679734 8840 hybrid_clock.cc:648] HybridClock initialized: now 1786208492679719 us; error 28 us; skew 500 ppm
Aug 08 17:01:32 dist-test-slave-1nl6 krb5kdc[8820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208492, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:32.682185 8840 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 17:01:32.682611 8840 webserver.cc:460] Webserver started at http://127.8.14.126:41433/ using document root <none> and password file <none>
I20260808 17:01:32.682832 8840 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:32.682900 8840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:32.683079 8840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:32.684094 8840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "c1cb206c184d48419157aab1f53e2d8f"
format_stamp: "Formatted at 2026-08-08 17:01:32 on dist-test-slave-1nl6"
server_key: "913f841d16cf4eecf009d441096766b2"
server_key_iv: "96b8e810e27501153419657ff5c57e45"
server_key_version: "encryptionkey@0"
I20260808 17:01:32.684468 8840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "c1cb206c184d48419157aab1f53e2d8f"
format_stamp: "Formatted at 2026-08-08 17:01:32 on dist-test-slave-1nl6"
server_key: "913f841d16cf4eecf009d441096766b2"
server_key_iv: "96b8e810e27501153419657ff5c57e45"
server_key_version: "encryptionkey@0"
I20260808 17:01:32.685778 8840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 17:01:32.686480 8868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:32.686653 8840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 17:01:32.686757 8840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "c1cb206c184d48419157aab1f53e2d8f"
format_stamp: "Formatted at 2026-08-08 17:01:32 on dist-test-slave-1nl6"
server_key: "913f841d16cf4eecf009d441096766b2"
server_key_iv: "96b8e810e27501153419657ff5c57e45"
server_key_version: "encryptionkey@0"
I20260808 17:01:32.686842 8840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:32.695907 8840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:32.698191 8840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:32.698369 8840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:32.702524 8840 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:37083
I20260808 17:01:32.702561 8937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:37083 every 8 connection(s)
I20260808 17:01:32.702963 8840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:32.704007 8938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:32.705554 8938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:01:32 dist-test-slave-1nl6 krb5kdc[8820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208492, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:01:32.712087 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 8840
I20260808 17:01:32.712247 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:32.712468 8249 external_mini_cluster.cc:1428] Setting key bb15ae373ce564c6da23fe6b234d4c98
I20260808 17:01:32.715219 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.706219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:37083 (local address 127.0.0.1:44396)
0808 17:01:32.706528 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.706556 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:32.707079 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:32.707342 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:32.707344 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:32.707580 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:32.707731 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:32.707735 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:32.708719 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:32.708720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:32.709400 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:32.709402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:32.709876 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:32.709877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:32.709932 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:32.710374 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:32.710382 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:32.711747 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:32.713641 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:32.713648 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:32.713650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:32.713848 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:32.714102 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:32.714103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:32.714103 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:32.714193 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:32.714546 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:32.714552 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:32.714612 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 17:01:32.714720 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 17:01:32.714983 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":94,"thread_start_us":122,"threads_started":1}
I20260808 17:01:32.715221 8942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.706284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44396 (local address 127.8.14.126:37083)
0808 17:01:32.706471 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:32.706478 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:32.706657 (+ 179us) server_negotiation.cc:407] Connection header received
0808 17:01:32.707207 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:32.707209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:32.707225 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:32.707303 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:32.707823 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:32.708632 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:32.709478 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:32.709778 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:32.709855 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:32.711845 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:32.711853 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:32.711854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:32.711860 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:32.713535 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:32.713945 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:32.713948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:32.713948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:32.714030 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:32.714270 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:32.714271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:32.714272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:32.714471 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:32.714566 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:32.714754 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 17:01:32.714894 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":105,"threads_started":1}
I20260808 17:01:32.715936 8938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:01:32.716282 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.716169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:42321 (local address 127.0.0.1:48186)
0808 17:01:32.716219 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.716263 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:42321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 17:01:32.716449 8873 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.14.125:42321: connect: Connection refused (error 111)
I20260808 17:01:32.716840 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.716776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:42321 (local address 127.0.0.1:48190)
0808 17:01:32.716801 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.716823 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:42321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260808 17:01:32.716950 8938 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.125:42321: Network error: Client connection negotiation failed: client connection to 127.8.14.125:42321: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.8.14.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.125.keytab.
Entry for principal kudu/127.8.14.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.125.keytab.
I20260808 17:01:32.737174 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.125:42321
--webserver_interface=127.8.14.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.125.keytab
--principal=kudu/127.8.14.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:35489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:37083,127.8.14.125:42321,127.8.14.124:46793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:01:32.766170 8938 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } attempt: 1
I20260808 17:01:32.766683 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.766547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:42321 (local address 127.0.0.1:48196)
0808 17:01:32.766614 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.766653 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:42321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 17:01:32.767134 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.766984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:42321 (local address 127.0.0.1:48212)
0808 17:01:32.767056 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.767100 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:42321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 17:01:32.767306 8938 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.125:42321: Network error: Client connection negotiation failed: client connection to 127.8.14.125:42321: connect: Connection refused (error 111)
W20260808 17:01:32.819646 8952 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:01:32.819855 8952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:32.819873 8952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:32.819893 8952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:32.821082 8952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:32.821126 8952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:32.821141 8952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:32.821153 8952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:32.821165 8952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:32.822640 8952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:35489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:37083,127.8.14.125:42321,127.8.14.124:46793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.125.keytab
--principal=kudu/127.8.14.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.125:42321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.14.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:32.822929 8952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:32.823179 8952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:32.827852 8960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:32.827907 8963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:32.828079 8952 server_base.cc:1034] running on GCE node
W20260808 17:01:32.827860 8959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:32.828423 8952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:32.828694 8952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:32.829839 8952 hybrid_clock.cc:648] HybridClock initialized: now 1786208492829829 us; error 29 us; skew 500 ppm
Aug 08 17:01:32 dist-test-slave-1nl6 krb5kdc[8820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208492, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:32.832480 8952 init.cc:377] Logged in from keytab as kudu/127.8.14.125@KRBTEST.COM (short username kudu)
I20260808 17:01:32.832903 8952 webserver.cc:460] Webserver started at http://127.8.14.125:45353/ using document root <none> and password file <none>
I20260808 17:01:32.833127 8952 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:32.833194 8952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:32.833357 8952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:32.834388 8952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-1/data/instance:
uuid: "9d803ebba648448e9b714fa6c3b17724"
format_stamp: "Formatted at 2026-08-08 17:01:32 on dist-test-slave-1nl6"
server_key: "e09ce75845a43760a9e3e115c2fe730f"
server_key_iv: "c00f3d514c1dd127b05330a25ebaf889"
server_key_version: "encryptionkey@0"
I20260808 17:01:32.834751 8952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-1/wal/instance:
uuid: "9d803ebba648448e9b714fa6c3b17724"
format_stamp: "Formatted at 2026-08-08 17:01:32 on dist-test-slave-1nl6"
server_key: "e09ce75845a43760a9e3e115c2fe730f"
server_key_iv: "c00f3d514c1dd127b05330a25ebaf889"
server_key_version: "encryptionkey@0"
I20260808 17:01:32.835495 8938 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } attempt: 2
I20260808 17:01:32.835886 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.835758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:42321 (local address 127.0.0.1:48216)
0808 17:01:32.835821 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.835856 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:42321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 17:01:32.836323 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.836196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:42321 (local address 127.0.0.1:48226)
0808 17:01:32.836248 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.836281 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:42321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 17:01:32.836447 8938 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.125:42321: Network error: Client connection negotiation failed: client connection to 127.8.14.125:42321: connect: Connection refused (error 111)
I20260808 17:01:32.836431 8952 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:01:32.837177 8970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:32.837380 8952 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 17:01:32.837481 8952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-1/wal
uuid: "9d803ebba648448e9b714fa6c3b17724"
format_stamp: "Formatted at 2026-08-08 17:01:32 on dist-test-slave-1nl6"
server_key: "e09ce75845a43760a9e3e115c2fe730f"
server_key_iv: "c00f3d514c1dd127b05330a25ebaf889"
server_key_version: "encryptionkey@0"
I20260808 17:01:32.837567 8952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:32.848217 8952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:32.850059 8952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:32.850229 8952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:32.854112 8952 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.125:42321
I20260808 17:01:32.854162 9050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.125:42321 every 8 connection(s)
I20260808 17:01:32.854521 8952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-1/data/info.pb
I20260808 17:01:32.855561 9051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:32.857036 9051 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:01:32.861958 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 8952
I20260808 17:01:32.862073 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-1/wal/instance
I20260808 17:01:32.862258 8249 external_mini_cluster.cc:1428] Setting key cab6cd726f8e1d4a83c9cb3fe8d45925
Aug 08 17:01:32 dist-test-slave-1nl6 krb5kdc[8820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208492, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.125@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:01:32.866627 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.857667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:37083 (local address 127.0.0.1:44404)
0808 17:01:32.857848 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.857906 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:32.858494 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:32.858784 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:32.858788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:32.859154 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:32.859339 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:32.859343 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:32.860025 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:32.860026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:32.860785 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:32.860787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:32.861202 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:32.861204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:32.861317 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:32.861718 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:32.861725 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:32.863501 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:32.865338 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:32.865345 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:32.865347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:32.865567 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:32.865789 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:32.865790 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:32.865790 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:32.865884 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:32.866170 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:32.866174 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:32.866279 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 17:01:32.866366 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 17:01:32.866479 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":90,"threads_started":1}
I20260808 17:01:32.866655 8942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.857720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44404 (local address 127.8.14.126:37083)
0808 17:01:32.857782 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:32.857785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:32.857995 (+ 210us) server_negotiation.cc:407] Connection header received
0808 17:01:32.858610 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:32.858611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:32.858635 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:32.858711 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:32.859437 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:32.859929 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:32.860874 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:32.861093 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:32.861173 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:32.863593 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:32.863594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:32.863594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:32.863601 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:32.865248 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:32.865654 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:32.865655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:32.865655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:32.865719 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:32.865950 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:32.865951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:32.865952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:32.866104 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:32.866175 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:32.866410 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 17:01:32.866505 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 17:01:32.867355 9051 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.14.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.124@KRBTEST.COM" created.
Aug 08 17:01:32 dist-test-slave-1nl6 krb5kdc[8820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208492, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.125@KRBTEST.COM for kudu/127.8.14.125@KRBTEST.COM
I20260808 17:01:32.877082 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.867631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:42321 (local address 127.0.0.1:48242)
0808 17:01:32.867691 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.867700 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:32.867744 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:32.868000 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:32.868001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:32.868142 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:32.868181 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:32.868183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:32.869004 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:32.869005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:32.869498 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:32.869499 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:32.869964 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:32.869965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:32.870029 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:32.870435 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:32.870440 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:32.874069 (+ 3629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:32.875950 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:32.875951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:32.875952 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:32.876184 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:32.876402 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:32.876402 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:32.876403 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:32.876474 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:32.876755 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:32.876755 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:32.876794 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 17:01:32.876863 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 17:01:32.876968 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 17:01:32.877144 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.867706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48242 (local address 127.8.14.125:42321)
0808 17:01:32.867869 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:32.867873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:32.867889 (+ 16us) server_negotiation.cc:407] Connection header received
0808 17:01:32.867906 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:32.867907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:32.867917 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:32.867999 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:32.868264 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:32.868915 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:32.869580 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:32.869887 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:32.869966 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:32.874168 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:32.874181 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:32.874181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:32.874186 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:32.875855 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:32.876280 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:32.876283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:32.876283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:32.876337 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:32.876531 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:32.876533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:32.876533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:32.876681 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:32.876778 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:32.876904 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 17:01:32.877021 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
I20260808 17:01:32.878170 9051 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:01:32.878527 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.878405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46793 (local address 127.0.0.1:37074)
0808 17:01:32.878453 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.878502 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 17:01:32.878662 8976 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46793: connect: Connection refused (error 111)
I20260808 17:01:32.879112 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.879016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46793 (local address 127.0.0.1:37076)
0808 17:01:32.879064 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.879088 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 17:01:32.879233 9051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.124:46793: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46793: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.124.keytab.
Entry for principal kudu/127.8.14.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.124.keytab.
I20260808 17:01:32.887861 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.124:46793
--webserver_interface=127.8.14.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.124.keytab
--principal=kudu/127.8.14.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:35489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:37083,127.8.14.125:42321,127.8.14.124:46793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:01:32.927598 8938 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } attempt: 3
I20260808 17:01:32.928467 9051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } attempt: 1
I20260808 17:01:32.928881 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.928745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46793 (local address 127.0.0.1:37090)
0808 17:01:32.928803 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.928843 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 17:01:32.929242 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.929126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46793 (local address 127.0.0.1:37104)
0808 17:01:32.929178 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.929212 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 17:01:32.929391 9051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.124:46793: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46793: connect: Connection refused (error 111)
Aug 08 17:01:32 dist-test-slave-1nl6 krb5kdc[8820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208492, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for kudu/127.8.14.125@KRBTEST.COM
I20260808 17:01:32.935128 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.928018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:42321 (local address 127.0.0.1:48254)
0808 17:01:32.928074 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.928091 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:32.928166 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:32.928339 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:32.928340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:32.928525 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:32.928627 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:32.928629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:32.929545 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:32.929546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:32.930058 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:32.930060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:32.930464 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:32.930465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:32.930545 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:32.931043 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:32.931048 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:32.932498 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:32.934101 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:32.934102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:32.934102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:32.934313 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:32.934499 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:32.934500 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:32.934501 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:32.934567 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:32.934785 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:32.934786 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:32.934836 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 17:01:32.934920 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 17:01:32.935036 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 17:01:32.935140 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.928090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48254 (local address 127.8.14.125:42321)
0808 17:01:32.928163 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:32.928165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:32.928176 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:01:32.928231 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:32.928232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:32.928242 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:32.928309 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:32.928720 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:32.929475 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:32.930147 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:32.930388 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:32.930466 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:32.932583 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:32.932584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:32.932584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:32.932590 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:32.934013 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:32.934388 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:32.934388 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:32.934389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:32.934438 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:32.934626 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:32.934627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:32.934627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:32.934726 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:32.934773 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:32.935023 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 17:01:32.935085 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 17:01:32.935555 8938 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:01:32.935891 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.935781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46793 (local address 127.0.0.1:37110)
0808 17:01:32.935831 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.935866 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 17:01:32.936204 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.936118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46793 (local address 127.0.0.1:37120)
0808 17:01:32.936162 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.936187 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 17:01:32.936309 8938 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.124:46793: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46793: connect: Connection refused (error 111)
I20260808 17:01:32.967471 8938 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } attempt: 1
I20260808 17:01:32.968031 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.967889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46793 (local address 127.0.0.1:37132)
0808 17:01:32.967960 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.968003 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 17:01:32.968437 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.968347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46793 (local address 127.0.0.1:37138)
0808 17:01:32.968395 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.968421 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 17:01:32.968567 8938 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.124:46793: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46793: connect: Connection refused (error 111)
W20260808 17:01:32.973341 9060 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:01:32.973536 9060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:32.973551 9060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:32.973619 9060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:32.974759 9060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:32.974799 9060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:32.974813 9060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:32.974825 9060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:32.974879 9060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:32.976423 9060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:35489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:37083,127.8.14.125:42321,127.8.14.124:46793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.124.keytab
--principal=kudu/127.8.14.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.124:46793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.14.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:32.976691 9060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:32.976943 9060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:32.981779 9070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:32.981931 9060 server_base.cc:1034] running on GCE node
W20260808 17:01:32.981794 9068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:32.981804 9066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:32.982388 9060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:32.982741 9060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:32.983901 9060 hybrid_clock.cc:648] HybridClock initialized: now 1786208492983878 us; error 42 us; skew 500 ppm
Aug 08 17:01:32 dist-test-slave-1nl6 krb5kdc[8820](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208492, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:32.986424 9060 init.cc:377] Logged in from keytab as kudu/127.8.14.124@KRBTEST.COM (short username kudu)
I20260808 17:01:32.986873 9060 webserver.cc:460] Webserver started at http://127.8.14.124:46785/ using document root <none> and password file <none>
I20260808 17:01:32.987162 9060 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:32.987236 9060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:32.987391 9060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:32.988461 9060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-2/data/instance:
uuid: "056fceceefa14feba60fe97fddb5ef6f"
format_stamp: "Formatted at 2026-08-08 17:01:32 on dist-test-slave-1nl6"
server_key: "07245819dd3152f659dc13eb92b45e51"
server_key_iv: "3e7021d74556cc2c9c8bc109dfdffe88"
server_key_version: "encryptionkey@0"
I20260808 17:01:32.988833 9060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-2/wal/instance:
uuid: "056fceceefa14feba60fe97fddb5ef6f"
format_stamp: "Formatted at 2026-08-08 17:01:32 on dist-test-slave-1nl6"
server_key: "07245819dd3152f659dc13eb92b45e51"
server_key_iv: "3e7021d74556cc2c9c8bc109dfdffe88"
server_key_version: "encryptionkey@0"
I20260808 17:01:32.990231 9060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:01:32.991089 9084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:32.991268 9060 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:32.991382 9060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-2/wal
uuid: "056fceceefa14feba60fe97fddb5ef6f"
format_stamp: "Formatted at 2026-08-08 17:01:32 on dist-test-slave-1nl6"
server_key: "07245819dd3152f659dc13eb92b45e51"
server_key_iv: "3e7021d74556cc2c9c8bc109dfdffe88"
server_key_version: "encryptionkey@0"
I20260808 17:01:32.991461 9060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:32.997610 9051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } attempt: 2
I20260808 17:01:32.998031 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.997902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46793 (local address 127.0.0.1:37152)
0808 17:01:32.997962 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.997994 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 17:01:32.998360 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:32.998271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46793 (local address 127.0.0.1:37166)
0808 17:01:32.998316 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:32.998339 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 17:01:32.998466 9051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.124:46793: Network error: Client connection negotiation failed: client connection to 127.8.14.124:46793: connect: Connection refused (error 111)
I20260808 17:01:33.003866 9060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:33.006292 9060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:33.006466 9060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:33.010542 9060 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.124:46793
I20260808 17:01:33.010589 9167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.124:46793 every 8 connection(s)
I20260808 17:01:33.011044 9060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-2/data/info.pb
I20260808 17:01:33.012092 9168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:33.013228 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 9060
I20260808 17:01:33.013351 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786208190438830-8249-0/minicluster-data/master-2/wal/instance
I20260808 17:01:33.013531 8249 external_mini_cluster.cc:1428] Setting key 2d0e7233f71b78dc73f639c1b89e747b
I20260808 17:01:33.013801 9168 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:01:33 dist-test-slave-1nl6 krb5kdc[8820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208492, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.124@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
Aug 08 17:01:33 dist-test-slave-1nl6 krb5kdc[8820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.124@KRBTEST.COM
I20260808 17:01:33.023757 8942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:33.014598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44416 (local address 127.8.14.126:37083)
0808 17:01:33.014652 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.014654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.014790 (+ 136us) server_negotiation.cc:407] Connection header received
0808 17:01:33.015009 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.015011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.015023 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.015082 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.015641 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.016121 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.016995 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.017190 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.017283 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:33.019667 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:33.019668 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:33.019669 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:33.019676 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:33.022161 (+ 2485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.022664 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.022666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.022666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.022722 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.023151 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.023152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.023152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.023281 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:33.023347 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:33.023585 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 17:01:33.023664 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:01:33.023872 9172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:33.014072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37170 (local address 127.8.14.124:46793)
0808 17:01:33.014262 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.014269 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.014294 (+ 25us) server_negotiation.cc:407] Connection header received
0808 17:01:33.014880 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.014887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.014956 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.015071 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.015697 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.016478 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.017200 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.017521 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.017614 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:33.020254 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:33.020263 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:33.020263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:33.020271 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:33.022226 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.022619 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.022622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.022622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.022728 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.022962 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.022963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.022963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.023255 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:33.023405 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:33.023510 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 17:01:33.023653 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":70,"threads_started":1}
I20260808 17:01:33.024075 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:33.014528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:37083 (local address 127.0.0.1:44416)
0808 17:01:33.014663 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:33.014705 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:33.014887 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:33.015104 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:33.015106 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:33.015388 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:33.015543 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:33.015547 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.016205 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.016206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:33.016902 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:33.016904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.017314 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.017315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:33.017443 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:33.017943 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:33.017953 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:33.019571 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:33.022262 (+ 2691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:33.022269 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:33.022271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:33.022555 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:33.022989 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:33.022989 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:33.022990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:33.023079 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:33.023376 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:33.023381 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:33.023443 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 17:01:33.023852 (+ 409us) client_negotiation.cc:240] Negotiation successful
0808 17:01:33.023972 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
I20260808 17:01:33.024602 9168 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 17:01:33 dist-test-slave-1nl6 krb5kdc[8820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208492, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.124@KRBTEST.COM for kudu/127.8.14.125@KRBTEST.COM
Aug 08 17:01:33 dist-test-slave-1nl6 krb5kdc[8820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:01:33.031708 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:33.024855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:42321 (local address 127.0.0.1:48256)
0808 17:01:33.024931 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:33.024940 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:33.024981 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:33.025129 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:33.025129 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:33.025268 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:33.025303 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:33.025305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.025920 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.025921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:33.026405 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:33.026406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.026791 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.026792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:33.026866 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:33.027298 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:33.027303 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:33.028652 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:33.030648 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:33.030649 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:33.030649 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:33.030862 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:33.031108 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:33.031109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:33.031110 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:33.031177 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:33.031447 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:33.031448 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:33.031482 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 17:01:33.031535 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 17:01:33.031601 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 17:01:33.031730 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:33.024921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48256 (local address 127.8.14.125:42321)
0808 17:01:33.024965 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.024967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.024976 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:01:33.025043 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.025043 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.025052 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.025126 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.025385 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.025841 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.026497 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.026702 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.026789 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:33.028749 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:33.028751 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:33.028751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:33.028759 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:33.030569 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.030991 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.030992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.030992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.031044 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.031244 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.031245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.031245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.031368 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:33.031440 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:33.031576 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 17:01:33.031657 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:01:33.032090 9168 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:01:33.032578 8942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:33.024771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44422 (local address 127.8.14.126:37083)
0808 17:01:33.024815 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.024817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.024867 (+ 50us) server_negotiation.cc:407] Connection header received
0808 17:01:33.024953 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.024954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.024962 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.025019 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.025474 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.025923 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.026604 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.026810 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.026882 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:33.029556 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:33.029557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:33.029557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:33.029565 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:33.031305 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.031675 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.031676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.031676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.031734 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.032033 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.032034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.032035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.032180 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:33.032366 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:33.032404 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 17:01:33.032484 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 08 17:01:33 dist-test-slave-1nl6 krb5kdc[8820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208492, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.124@KRBTEST.COM for kudu/127.8.14.124@KRBTEST.COM
Aug 08 17:01:33 dist-test-slave-1nl6 krb5kdc[8820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.125@KRBTEST.COM
I20260808 17:01:33.038866 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:33.032327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46793 (local address 127.0.0.1:37182)
0808 17:01:33.032397 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:33.032406 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:33.032447 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:33.032590 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:33.032591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:33.032750 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:33.032776 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:33.032777 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.033529 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.033530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:33.033999 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:33.034000 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.034361 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.034362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:33.034409 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:33.034804 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:33.034809 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:33.036097 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:33.037835 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:33.037836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:33.037836 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:33.038048 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:33.038289 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:33.038289 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:33.038289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:33.038342 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:33.038624 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:33.038625 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:33.038670 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 17:01:33.038738 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 17:01:33.038809 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 17:01:33.039039 9172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:33.032365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37182 (local address 127.8.14.124:46793)
0808 17:01:33.032397 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.032398 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.032433 (+ 35us) server_negotiation.cc:407] Connection header received
0808 17:01:33.032499 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.032500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.032509 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.032558 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.032847 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.033452 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.034089 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.034291 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.034364 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:33.036187 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:33.036188 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:33.036189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:33.036195 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:33.037753 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.038138 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.038139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.038139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.038202 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.038411 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.038413 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.038413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.038530 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:33.038616 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:33.038792 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 17:01:33.038880 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 17:01:33.040745 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:33.033273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.8.14.125:42321)
0808 17:01:33.033319 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.033321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.033328 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:01:33.033348 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.033349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.033357 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.033419 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.033878 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.034295 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.034990 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.035174 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.035246 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:33.037709 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:33.037710 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:33.037710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:33.037716 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:33.039657 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.040009 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.040010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.040010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.040059 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.040267 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.040267 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.040268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.040385 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:33.040492 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:33.040580 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 17:01:33.040669 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 17:01:33.040831 9168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f: Bootstrap starting.
I20260808 17:01:33.041960 9168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:33.042224 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:33.042315 9168 log.cc:824] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:33.043170 9168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f: No bootstrap required, opened a new log
I20260808 17:01:33.043730 8938 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } attempt: 2
I20260808 17:01:33.043347 8904 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c1cb206c184d48419157aab1f53e2d8f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44422
I20260808 17:01:33.045167 9168 raft_consensus.cc:348] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } }
I20260808 17:01:33.045320 9168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 056fceceefa14feba60fe97fddb5ef6f, State: Initialized, Role: FOLLOWER
I20260808 17:01:33.045404 9168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } }
I20260808 17:01:33.045677 9176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } } }
I20260808 17:01:33.045760 9168 sys_catalog.cc:564] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:33.045768 9176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f [sys.catalog]: This master's current role is: FOLLOWER
Aug 08 17:01:33 dist-test-slave-1nl6 krb5kdc[8820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208492, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for kudu/127.8.14.124@KRBTEST.COM
W20260808 17:01:33.047583 9190 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:01:33.049328 9190 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:01:33.051455 8941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:33.043970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46793 (local address 127.0.0.1:37184)
0808 17:01:33.044286 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:33.044297 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:33.044380 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:33.044492 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:33.044493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:33.044641 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:33.044673 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:33.044674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.045279 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.045280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:33.046079 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:33.046082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.046447 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.046448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:33.046505 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:33.046971 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:33.046975 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:33.048383 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:33.050227 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:33.050228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:33.050229 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:33.050474 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:33.050696 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:33.050697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:33.050698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:33.050767 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:33.051122 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:33.051124 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:33.051169 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 17:01:33.051254 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 17:01:33.051351 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301}
I20260808 17:01:33.051456 9172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:33.044043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37184 (local address 127.8.14.124:46793)
0808 17:01:33.044088 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.044090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.044369 (+ 279us) server_negotiation.cc:407] Connection header received
0808 17:01:33.044437 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.044438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.044447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.044491 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.044757 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.045191 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.046151 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.046368 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.046447 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:33.048475 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:33.048477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:33.048477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:33.048484 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:33.050126 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.050561 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.050562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.050562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.050621 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.050833 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.050834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.050834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.051010 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:33.051117 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:33.051287 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 17:01:33.051367 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:01:33.049407 9060 master_runner.cc:186] Error getting master registration for 127.8.14.126:37083: OK, instance_id { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" instance_seqno: 1786208492701320 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 17:01:33.051983 9060 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 17:01:33.052968 8938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f: Bootstrap starting.
I20260808 17:01:33.053694 8938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:33.053995 8938 log.cc:824] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:33.054894 8938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f: No bootstrap required, opened a new log
I20260808 17:01:33.056282 8938 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } }
I20260808 17:01:33.056433 8938 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1cb206c184d48419157aab1f53e2d8f, State: Initialized, Role: FOLLOWER
I20260808 17:01:33.056520 8938 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } }
I20260808 17:01:33.056823 8938 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:33.056818 9194 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } } }
I20260808 17:01:33.056866 8904 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 17:01:33.056927 9194 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:01:33.056957 8904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:33.057719 8904 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } }
I20260808 17:01:33.057932 8904 leader_election.cc:288] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [CANDIDATE]: Term 1 election: Requested vote from peers 9d803ebba648448e9b714fa6c3b17724 (127.8.14.125:42321), 056fceceefa14feba60fe97fddb5ef6f (127.8.14.124:46793)
I20260808 17:01:33.058858 9124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1cb206c184d48419157aab1f53e2d8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "056fceceefa14feba60fe97fddb5ef6f"
I20260808 17:01:33.059015 9124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:33.059955 9124 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1cb206c184d48419157aab1f53e2d8f in term 1.
I20260808 17:01:33.060186 8873 leader_election.cc:302] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 056fceceefa14feba60fe97fddb5ef6f, c1cb206c184d48419157aab1f53e2d8f; no voters:
I20260808 17:01:33.060329 9194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:33.060478 9194 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [term 1 LEADER]: Becoming Leader. State: Replica: c1cb206c184d48419157aab1f53e2d8f, State: Running, Role: LEADER
I20260808 17:01:33.061044 9194 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } }
I20260808 17:01:33.061491 9194 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1cb206c184d48419157aab1f53e2d8f. Latest consensus state: current_term: 1 leader_uuid: "c1cb206c184d48419157aab1f53e2d8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } } }
I20260808 17:01:33.061587 9194 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:33.062224 9220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:33.060089 8840 master_runner.cc:186] Error getting master registration for 127.8.14.125:42321: OK, instance_id { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" instance_seqno: 1786208492853030 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 17:01:33.062567 8840 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 17:01:33.062733 9220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:33.068517 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:33.059777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48288 (local address 127.8.14.125:42321)
0808 17:01:33.059822 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.059824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.060457 (+ 633us) server_negotiation.cc:407] Connection header received
0808 17:01:33.060593 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.060594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.060607 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.060637 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.061498 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.062201 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.063413 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.063717 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.063770 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:33.065348 (+ 1578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:33.065350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:33.065350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:33.065357 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:33.067115 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.067506 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.067508 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.067508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.067568 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.067812 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.067813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.067814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.067960 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:33.068041 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:33.068317 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 17:01:33.068405 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 17:01:33.071028 9124 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f [term 1 FOLLOWER]: Refusing update from remote peer c1cb206c184d48419157aab1f53e2d8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:33.071389 9194 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:33.072834 9176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1cb206c184d48419157aab1f53e2d8f. Latest consensus state: current_term: 1 leader_uuid: "c1cb206c184d48419157aab1f53e2d8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } } }
I20260808 17:01:33.072937 9176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:01:33.073635 8942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:33.063696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44426 (local address 127.8.14.126:37083)
0808 17:01:33.063759 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.063761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.064024 (+ 263us) server_negotiation.cc:407] Connection header received
0808 17:01:33.064114 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.064115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.064124 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.064153 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.064893 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.065598 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.066617 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.066923 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.066969 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:33.068419 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:33.068421 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:33.068421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:33.068428 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:33.070107 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.070527 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.070528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.070529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.070582 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.070772 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.070772 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.070773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.072579 (+ 1806us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:33.072664 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:33.073467 (+ 803us) server_negotiation.cc:299] Negotiation successful
0808 17:01:33.073545 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 17:01:33.074415 9172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:33.060140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37200 (local address 127.8.14.124:46793)
0808 17:01:33.062653 (+ 2513us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.062656 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.063076 (+ 420us) server_negotiation.cc:407] Connection header received
0808 17:01:33.063214 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.063215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.063239 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.063268 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.064636 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.065376 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.067071 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.067359 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.067404 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:33.070654 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:33.070655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:33.070656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:33.070664 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:33.072378 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.073287 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.073288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.073288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.073361 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.073602 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.073603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.073603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.073765 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:33.073877 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:33.074213 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 17:01:33.074300 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2488}
I20260808 17:01:33.088168 9176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1cb206c184d48419157aab1f53e2d8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } } }
I20260808 17:01:33.088171 9194 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1cb206c184d48419157aab1f53e2d8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } } }
I20260808 17:01:33.088267 9176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:01:33.088289 9194 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:33.088572 9225 mvcc.cc:205] Tried to move back new op lower bound from 7316309987614085120 to 7316309987578863616. Current Snapshot: MvccSnapshot[applied={T|T < 7316309987614085120}]
I20260808 17:01:33.089238 9220 catalog_manager.cc:1347] Generated new cluster ID: 9f1268019c5b4dd5ace80c4fe6c764b2
I20260808 17:01:33.089290 9220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:33.089584 9051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } attempt: 3
Aug 08 17:01:33 dist-test-slave-1nl6 krb5kdc[8820](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208492, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.125@KRBTEST.COM for kudu/127.8.14.124@KRBTEST.COM
I20260808 17:01:33.106769 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:33.089855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:46793 (local address 127.0.0.1:37206)
0808 17:01:33.089907 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:33.089922 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:33.089973 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:33.099148 (+ 9175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:33.099149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:33.099342 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:33.099388 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:33.099390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.100245 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.100246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:33.101016 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:33.101018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.101518 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.101519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:33.101580 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:33.102115 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:33.102120 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:33.103716 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:33.105662 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:33.105663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:33.105663 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:33.105874 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:33.106121 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:33.106122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:33.106122 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:33.106187 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:33.106470 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:33.106471 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:33.106524 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 17:01:33.106579 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 17:01:33.106669 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 17:01:33.107259 9172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:33.098942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37206 (local address 127.8.14.124:46793)
0808 17:01:33.098999 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.099001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.099010 (+ 9us) server_negotiation.cc:407] Connection header received
0808 17:01:33.099032 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.099033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.099044 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.099097 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.099458 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.100158 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.101115 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:33.101406 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:33.101454 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:33.103828 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:33.103829 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:33.103829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:33.103851 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:33.105558 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.105954 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.105955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.105956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.106048 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:33.106253 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:33.106254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:33.106254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:33.106380 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:33.106430 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:33.107107 (+ 677us) server_negotiation.cc:299] Negotiation successful
0808 17:01:33.107177 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:01:33.109000 9051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724: Bootstrap starting.
I20260808 17:01:33.109686 9051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:33.109961 9051 log.cc:824] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:33.115566 9051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724: No bootstrap required, opened a new log
I20260808 17:01:33.117573 9051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } }
I20260808 17:01:33.117784 9051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d803ebba648448e9b714fa6c3b17724, State: Initialized, Role: FOLLOWER
I20260808 17:01:33.117990 9051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } }
I20260808 17:01:33.119336 9013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1cb206c184d48419157aab1f53e2d8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9d803ebba648448e9b714fa6c3b17724"
I20260808 17:01:33.119393 9012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:33.120462 9012 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724 [term 1 FOLLOWER]: Refusing update from remote peer c1cb206c184d48419157aab1f53e2d8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:33.120760 9218 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:33.121693 9013 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1cb206c184d48419157aab1f53e2d8f in term 1.
I20260808 17:01:33.119297 9051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 17:01:33.125229 9256 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:01:33.125329 9256 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:01:33.125623 9244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1cb206c184d48419157aab1f53e2d8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } } }
I20260808 17:01:33.125765 9244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:01:33.125877 9244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1cb206c184d48419157aab1f53e2d8f. Latest consensus state: current_term: 1 leader_uuid: "c1cb206c184d48419157aab1f53e2d8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } } }
I20260808 17:01:33.125990 9244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:01:33.126492 9244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1cb206c184d48419157aab1f53e2d8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } } }
I20260808 17:01:33.126634 9244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:01:33.127457 9220 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:33.128075 9220 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:33.131775 9194 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1cb206c184d48419157aab1f53e2d8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cb206c184d48419157aab1f53e2d8f" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 37083 } } peers { permanent_uuid: "9d803ebba648448e9b714fa6c3b17724" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 42321 } } peers { permanent_uuid: "056fceceefa14feba60fe97fddb5ef6f" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 46793 } } }
I20260808 17:01:33.131877 9194 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:33.131935 9236 mvcc.cc:205] Tried to move back new op lower bound from 7316309987614085120 to 7316309987578863616. Current Snapshot: MvccSnapshot[applied={T|T < 7316309987614085120}]
I20260808 17:01:33.133788 9220 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1cb206c184d48419157aab1f53e2d8f: Generated new TSK 0
I20260808 17:01:33.134011 9220 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 17:01:33.172072 9057 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:33.170368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48290 (local address 127.8.14.125:42321)
0808 17:01:33.170416 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.170418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.171410 (+ 992us) server_negotiation.cc:407] Connection header received
0808 17:01:33.171519 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.171520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.171531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.171564 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.172005 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48290: BlockingRecv error: recv got EOF from 127.0.0.1:48290 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 17:01:33.173185 8942 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:33.171124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44436 (local address 127.8.14.126:37083)
0808 17:01:33.171163 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.171165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.172530 (+ 1365us) server_negotiation.cc:407] Connection header received
0808 17:01:33.172673 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.172674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.172684 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.172713 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.173123 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44436: BlockingRecv error: recv got EOF from 127.0.0.1:44436 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 17:01:33.173736 9172 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:33.170734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37210 (local address 127.8.14.124:46793)
0808 17:01:33.170772 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.170775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.172399 (+ 1624us) server_negotiation.cc:407] Connection header received
0808 17:01:33.172775 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.172776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.172787 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.172816 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.173682 (+ 866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37210: BlockingRecv error: recv got EOF from 127.0.0.1:37210 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 17:01:33.176133 9057 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:33.175456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48300 (local address 127.8.14.125:42321)
0808 17:01:33.175487 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.175489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.175661 (+ 172us) server_negotiation.cc:407] Connection header received
0808 17:01:33.175756 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.175757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.175774 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.175801 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.176089 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48300: BlockingRecv error: recv got EOF from 127.0.0.1:48300 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 17:01:33.179380 9057 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:33.178568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48314 (local address 127.8.14.125:42321)
0808 17:01:33.178600 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.178601 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.178831 (+ 230us) server_negotiation.cc:407] Connection header received
0808 17:01:33.178983 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.178984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.178994 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.179022 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.179340 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48314: BlockingRecv error: recv got EOF from 127.0.0.1:48314 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 17:01:33.183234 9057 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:33.182488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48324 (local address 127.8.14.125:42321)
0808 17:01:33.182521 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.182522 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.182753 (+ 231us) server_negotiation.cc:407] Connection header received
0808 17:01:33.182841 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.182841 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.182850 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.182877 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.183193 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48324: BlockingRecv error: recv got EOF from 127.0.0.1:48324 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 17:01:33.189064 9057 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:33.188335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48332 (local address 127.8.14.125:42321)
0808 17:01:33.188369 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.188371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.188604 (+ 233us) server_negotiation.cc:407] Connection header received
0808 17:01:33.188694 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.188695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.188704 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.188731 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.189023 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48332: BlockingRecv error: recv got EOF from 127.0.0.1:48332 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 17:01:33.199309 9057 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:33.198172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48340 (local address 127.8.14.125:42321)
0808 17:01:33.198209 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.198210 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.198459 (+ 249us) server_negotiation.cc:407] Connection header received
0808 17:01:33.198550 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.198551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.198561 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.198589 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.199249 (+ 660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48340: BlockingRecv error: recv got EOF from 127.0.0.1:48340 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 17:01:33.217798 9057 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:33.216796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48348 (local address 127.8.14.125:42321)
0808 17:01:33.216843 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.216845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.217159 (+ 314us) server_negotiation.cc:407] Connection header received
0808 17:01:33.217271 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.217272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.217284 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.217329 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.217742 (+ 413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48348: BlockingRecv error: recv got EOF from 127.0.0.1:48348 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 17:01:33.252389 9057 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:33.251360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48356 (local address 127.8.14.125:42321)
0808 17:01:33.251409 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.251411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.251698 (+ 287us) server_negotiation.cc:407] Connection header received
0808 17:01:33.251814 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.251816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.251829 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.251862 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.252329 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48356: BlockingRecv error: recv got EOF from 127.0.0.1:48356 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 17:01:33.318686 9057 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:33.317933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48360 (local address 127.8.14.125:42321)
0808 17:01:33.318010 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.318012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.318148 (+ 136us) server_negotiation.cc:407] Connection header received
0808 17:01:33.318212 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.318213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.318224 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.318278 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.318621 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48360: BlockingRecv error: recv got EOF from 127.0.0.1:48360 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 17:01:33.448589 9057 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:33.447838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48368 (local address 127.8.14.125:42321)
0808 17:01:33.447913 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.447915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.448048 (+ 133us) server_negotiation.cc:407] Connection header received
0808 17:01:33.448099 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.448100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.448111 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.448165 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.448505 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48368: BlockingRecv error: recv got EOF from 127.0.0.1:48368 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 17:01:33.706444 9057 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:33.705709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48370 (local address 127.8.14.125:42321)
0808 17:01:33.705779 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:33.705782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:33.705863 (+ 81us) server_negotiation.cc:407] Connection header received
0808 17:01:33.705950 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:33.705952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:33.705964 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:33.706037 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:33.706399 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48370: BlockingRecv error: recv got EOF from 127.0.0.1:48370 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 17:01:34.049800 9190 catalog_manager.cc:1259] Loaded cluster ID: 9f1268019c5b4dd5ace80c4fe6c764b2
I20260808 17:01:34.049870 9190 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f: loading cluster ID for follower catalog manager: success
I20260808 17:01:34.050591 9190 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f: acquiring CA information for follower catalog manager: success
I20260808 17:01:34.050871 9190 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 056fceceefa14feba60fe97fddb5ef6f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 17:01:34.125845 9256 catalog_manager.cc:1259] Loaded cluster ID: 9f1268019c5b4dd5ace80c4fe6c764b2
I20260808 17:01:34.125923 9256 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724: loading cluster ID for follower catalog manager: success
I20260808 17:01:34.126688 9256 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724: acquiring CA information for follower catalog manager: success
I20260808 17:01:34.126969 9256 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9d803ebba648448e9b714fa6c3b17724: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 17:01:34.222219 9359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:34.219717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48386 (local address 127.8.14.125:42321)
0808 17:01:34.219820 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:34.219822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:34.219865 (+ 43us) server_negotiation.cc:407] Connection header received
0808 17:01:34.219935 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:34.219936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:34.220005 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:34.220090 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:34.220260 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:34.220765 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:34.221500 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:34.221683 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:34.221733 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:34.221886 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:34.221993 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:34.222051 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:34.222086 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 17:01:34.222104 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":57,"threads_started":1}
I20260808 17:01:34.226563 9367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:34.224294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37216 (local address 127.8.14.124:46793)
0808 17:01:34.224391 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:34.224393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:34.224403 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:01:34.224451 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:34.224452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:34.224483 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:34.224549 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:34.224615 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:34.225186 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:34.225861 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:34.226057 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:34.226110 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:34.226234 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:34.226331 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:34.226387 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:34.226436 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 17:01:34.226453 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":55,"threads_started":1}
I20260808 17:01:34.230822 9359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:34.228085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48398 (local address 127.8.14.125:42321)
0808 17:01:34.228121 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:34.228123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:34.228360 (+ 237us) server_negotiation.cc:407] Connection header received
0808 17:01:34.228392 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:34.228393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:34.228405 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:34.228469 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:34.228631 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:34.229310 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:34.230131 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:34.230338 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:34.230422 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:34.230576 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:34.230606 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:34.230632 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:34.230727 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 17:01:34.230746 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:01:34.231396 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:34.228061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44444 (local address 127.8.14.126:37083)
0808 17:01:34.228351 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:34.228353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:34.228361 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:01:34.228411 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:34.228412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:34.228472 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:34.228542 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:34.228694 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:34.229222 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:34.230197 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:34.230506 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:34.230572 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:34.230767 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:34.230844 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:34.231124 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:34.231279 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 17:01:34.231305 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":44,"threads_started":1}
I20260808 17:01:34.231542 9367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:34.228457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37226 (local address 127.8.14.124:46793)
0808 17:01:34.228496 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:34.228498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:34.228579 (+ 81us) server_negotiation.cc:407] Connection header received
0808 17:01:34.228636 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:34.228637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:34.228647 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:34.228691 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:34.228871 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:34.229523 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:34.230537 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:34.230820 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:34.230887 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:34.231172 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:34.231202 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:34.231265 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:34.231427 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 17:01:34.231444 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 17:01:34.232986 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 8840
I20260808 17:01:34.245568 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 8952
I20260808 17:01:34.256330 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 9060
2026-08-08T17:01:34Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3021 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 17:01:34 dist-test-slave-1nl6 krb5kdc[9384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 17:01:34 dist-test-slave-1nl6 krb5kdc[9384](info): set up 2 sockets
Aug 08 17:01:34 dist-test-slave-1nl6 krb5kdc[9384](info): commencing operation
krb5kdc: starting...
W20260808 17:01:35.466140 8249 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/test-user.keytab.
Aug 08 17:01:35 dist-test-slave-1nl6 krb5kdc[9384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T17:01:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T17:01:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.14.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
Entry for principal kudu/127.8.14.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab.
I20260808 17:01:35.583693 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:34873
--webserver_interface=127.8.14.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:46269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:34873,127.8.14.125:46603,127.8.14.124:42869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 17:01:35.665570 9400 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:01:35.665784 9400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:35.665802 9400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:35.665822 9400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:35.666970 9400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:35.667017 9400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:35.667032 9400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:35.667044 9400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:35.667057 9400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:35.668558 9400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:46269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:34873,127.8.14.125:46603,127.8.14.124:42869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.126.keytab
--principal=kudu/127.8.14.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.126:34873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.14.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:35.668771 9400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:35.668949 9400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:35.673954 9409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:35.673985 9400 server_base.cc:1034] running on GCE node
W20260808 17:01:35.673931 9413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:35.673941 9411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:35.674389 9400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:35.674697 9400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:35.675868 9400 hybrid_clock.cc:648] HybridClock initialized: now 1786208495675844 us; error 54 us; skew 500 ppm
Aug 08 17:01:35 dist-test-slave-1nl6 krb5kdc[9384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208495, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:35.683513 9400 init.cc:377] Logged in from keytab as kudu/127.8.14.126@KRBTEST.COM (short username kudu)
I20260808 17:01:35.684028 9400 webserver.cc:460] Webserver started at http://127.8.14.126:35689/ using document root <none> and password file <none>
I20260808 17:01:35.684265 9400 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:35.684338 9400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:35.684501 9400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:35.685570 9400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-0/data/instance:
uuid: "90f8c02b96c34c93a397b17a27d21dc5"
format_stamp: "Formatted at 2026-08-08 17:01:35 on dist-test-slave-1nl6"
server_key: "ca5bd27b08ec303eb1f7bf35d5f37808"
server_key_iv: "e6824cead3414d58a6e0f5c26b5d059d"
server_key_version: "encryptionkey@0"
I20260808 17:01:35.685966 9400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-0/wal/instance:
uuid: "90f8c02b96c34c93a397b17a27d21dc5"
format_stamp: "Formatted at 2026-08-08 17:01:35 on dist-test-slave-1nl6"
server_key: "ca5bd27b08ec303eb1f7bf35d5f37808"
server_key_iv: "e6824cead3414d58a6e0f5c26b5d059d"
server_key_version: "encryptionkey@0"
I20260808 17:01:35.687455 9400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 17:01:35.688220 9422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:35.688385 9400 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:35.688500 9400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-0/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-0/wal
uuid: "90f8c02b96c34c93a397b17a27d21dc5"
format_stamp: "Formatted at 2026-08-08 17:01:35 on dist-test-slave-1nl6"
server_key: "ca5bd27b08ec303eb1f7bf35d5f37808"
server_key_iv: "e6824cead3414d58a6e0f5c26b5d059d"
server_key_version: "encryptionkey@0"
I20260808 17:01:35.688578 9400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:35.703588 9400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:35.706055 9400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:35.706236 9400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:35.710775 9400 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.126:34873
I20260808 17:01:35.710811 9494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.126:34873 every 8 connection(s)
I20260808 17:01:35.711244 9400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-0/data/info.pb
I20260808 17:01:35.712339 9495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:35.713956 9495 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:01:35.718571 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 9400
I20260808 17:01:35.718695 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-0/wal/instance
I20260808 17:01:35.718984 8249 external_mini_cluster.cc:1428] Setting key e071f85122c61a149bdd951fffd95222
Aug 08 17:01:35 dist-test-slave-1nl6 krb5kdc[9384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208495, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:01:35.724709 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.714709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:34873 (local address 127.0.0.1:39756)
0808 17:01:35.714882 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.714951 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:35.715450 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:35.715761 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:35.715763 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:35.716052 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:35.716212 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:35.716217 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:35.717361 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:35.717362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:35.718080 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:35.718083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:35.718650 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:35.718651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:35.718718 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:35.719327 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:35.719336 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:35.720972 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:35.723126 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:35.723135 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:35.723138 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:35.723378 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:35.723680 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:35.723681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:35.723681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:35.723764 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:35.724247 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:35.724253 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:35.724313 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 17:01:35.724389 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 17:01:35.724524 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260808 17:01:35.724798 9498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.714817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39756 (local address 127.8.14.126:34873)
0808 17:01:35.715005 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:35.715009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:35.715040 (+ 31us) server_negotiation.cc:407] Connection header received
0808 17:01:35.715628 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:35.715630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:35.715647 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:35.715754 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:35.716323 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:35.717130 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:35.718194 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:35.718522 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:35.718664 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:35.721073 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:35.721080 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:35.721081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:35.721088 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:35.723033 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:35.723467 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:35.723470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:35.723471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:35.723598 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:35.723833 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:35.723834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:35.723834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:35.724167 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:35.724280 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:35.724455 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 17:01:35.724617 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":161,"thread_start_us":136,"threads_started":1}
I20260808 17:01:35.725569 9495 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:01:35.725944 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.725845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:46603 (local address 127.0.0.1:43322)
0808 17:01:35.725882 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.725926 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:46603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260808 17:01:35.726081 9423 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.14.125:46603: connect: Connection refused (error 111)
I20260808 17:01:35.726786 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.726587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:46603 (local address 127.0.0.1:43324)
0808 17:01:35.726652 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.726764 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:46603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 17:01:35.726931 9495 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.125:46603: Network error: Client connection negotiation failed: client connection to 127.8.14.125:46603: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.8.14.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.125.keytab.
Entry for principal kudu/127.8.14.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.125.keytab.
I20260808 17:01:35.747545 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.125:46603
--webserver_interface=127.8.14.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.125.keytab
--principal=kudu/127.8.14.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:46269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:34873,127.8.14.125:46603,127.8.14.124:42869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:01:35.776314 9495 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } attempt: 1
I20260808 17:01:35.776938 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.776791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:46603 (local address 127.0.0.1:43330)
0808 17:01:35.776861 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.776908 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:46603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 17:01:35.777359 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.777262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:46603 (local address 127.0.0.1:43336)
0808 17:01:35.777315 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.777341 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:46603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 17:01:35.777485 9495 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.125:46603: Network error: Client connection negotiation failed: client connection to 127.8.14.125:46603: connect: Connection refused (error 111)
W20260808 17:01:35.831347 9502 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:01:35.831622 9502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:35.831652 9502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:35.831689 9502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:35.832871 9502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:35.832945 9502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:35.832970 9502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:35.832991 9502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:35.833020 9502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:35.834514 9502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:46269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:34873,127.8.14.125:46603,127.8.14.124:42869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.125.keytab
--principal=kudu/127.8.14.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.125:46603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.14.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:35.834928 9502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:35.835206 9502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:35.840425 9509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:35.840404 9512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:35.840409 9510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:35.840492 9502 server_base.cc:1034] running on GCE node
I20260808 17:01:35.840857 9502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:35.841116 9502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:35.842275 9502 hybrid_clock.cc:648] HybridClock initialized: now 1786208495842262 us; error 29 us; skew 500 ppm
Aug 08 17:01:35 dist-test-slave-1nl6 krb5kdc[9384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208495, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:35.844929 9502 init.cc:377] Logged in from keytab as kudu/127.8.14.125@KRBTEST.COM (short username kudu)
I20260808 17:01:35.845382 9502 webserver.cc:460] Webserver started at http://127.8.14.125:45921/ using document root <none> and password file <none>
I20260808 17:01:35.845672 9502 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:35.845638 9495 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } attempt: 2
I20260808 17:01:35.845757 9502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:35.845889 9502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:35.846033 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.845909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:46603 (local address 127.0.0.1:43352)
0808 17:01:35.845968 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.846003 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:46603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 17:01:35.846422 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.846317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:46603 (local address 127.0.0.1:43362)
0808 17:01:35.846373 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.846399 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.125:46603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 17:01:35.846573 9495 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.125:46603: Network error: Client connection negotiation failed: client connection to 127.8.14.125:46603: connect: Connection refused (error 111)
I20260808 17:01:35.847158 9502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-1/data/instance:
uuid: "61190ccb92714701a0a8c5081c03ab41"
format_stamp: "Formatted at 2026-08-08 17:01:35 on dist-test-slave-1nl6"
server_key: "0a2bbf377dce591bacfdde93ac63387d"
server_key_iv: "67d0353dbe2159ca4fcbf150368b23d9"
server_key_version: "encryptionkey@0"
I20260808 17:01:35.847473 9502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-1/wal/instance:
uuid: "61190ccb92714701a0a8c5081c03ab41"
format_stamp: "Formatted at 2026-08-08 17:01:35 on dist-test-slave-1nl6"
server_key: "0a2bbf377dce591bacfdde93ac63387d"
server_key_iv: "67d0353dbe2159ca4fcbf150368b23d9"
server_key_version: "encryptionkey@0"
I20260808 17:01:35.848784 9502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:35.849483 9521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:35.849663 9502 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 17:01:35.849736 9502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-1/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-1/wal
uuid: "61190ccb92714701a0a8c5081c03ab41"
format_stamp: "Formatted at 2026-08-08 17:01:35 on dist-test-slave-1nl6"
server_key: "0a2bbf377dce591bacfdde93ac63387d"
server_key_iv: "67d0353dbe2159ca4fcbf150368b23d9"
server_key_version: "encryptionkey@0"
I20260808 17:01:35.849788 9502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:35.859881 9502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:35.861642 9502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:35.861758 9502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:35.866062 9502 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.125:46603
I20260808 17:01:35.866140 9603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.125:46603 every 8 connection(s)
I20260808 17:01:35.866518 9502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-1/data/info.pb
I20260808 17:01:35.867600 9605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:35.869187 9605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:01:35.873564 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 9502
I20260808 17:01:35.873708 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-1/wal/instance
I20260808 17:01:35.873971 8249 external_mini_cluster.cc:1428] Setting key 2001951d57e4733186d7f4b986491257
Aug 08 17:01:35 dist-test-slave-1nl6 krb5kdc[9384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208495, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.125@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
I20260808 17:01:35.878880 9498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.869994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39762 (local address 127.8.14.126:34873)
0808 17:01:35.870078 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:35.870099 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:35.870221 (+ 122us) server_negotiation.cc:407] Connection header received
0808 17:01:35.870671 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:35.870672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:35.870685 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:35.870752 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:35.871475 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:35.871971 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:35.872939 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:35.873141 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:35.873217 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:35.875608 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:35.875610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:35.875610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:35.875619 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:35.877290 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:35.877725 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:35.877726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:35.877727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:35.877800 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:35.878051 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:35.878052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:35.878052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:35.878252 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:35.878334 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:35.878598 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 17:01:35.878723 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 17:01:35.878888 9609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.869886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:34873 (local address 127.0.0.1:39762)
0808 17:01:35.870078 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.870120 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:35.870581 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:35.870832 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:35.870836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:35.871158 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:35.871361 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:35.871366 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:35.872058 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:35.872058 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:35.872859 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:35.872861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:35.873238 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:35.873239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:35.873347 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:35.873981 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:35.873989 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:35.875495 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:35.877379 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:35.877386 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:35.877388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:35.877607 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:35.877871 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:35.877872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:35.877872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:35.877978 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:35.878325 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:35.878330 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:35.878460 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 17:01:35.878540 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 17:01:35.878665 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":113,"threads_started":1}
I20260808 17:01:35.879791 9605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.14.124@KRBTEST.COM; defaulting to no policy
Aug 08 17:01:35 dist-test-slave-1nl6 krb5kdc[9384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208495, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.125@KRBTEST.COM for kudu/127.8.14.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.14.124@KRBTEST.COM" created.
I20260808 17:01:35.888324 9609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.880077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:46603 (local address 127.0.0.1:43378)
0808 17:01:35.880157 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.880171 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:35.880225 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:35.880459 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:35.880460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:35.880659 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:35.880755 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:35.880757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:35.881711 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:35.881712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:35.882684 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:35.882687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:35.883253 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:35.883255 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:35.883341 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:35.883762 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:35.883768 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:35.885235 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:35.887228 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:35.887229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:35.887229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:35.887398 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:35.887617 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:35.887618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:35.887618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:35.887672 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:35.887982 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:35.887983 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:35.888021 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 17:01:35.888090 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 17:01:35.888195 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 17:01:35.888345 9612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.880134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43378 (local address 127.8.14.125:46603)
0808 17:01:35.880298 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:35.880303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:35.880321 (+ 18us) server_negotiation.cc:407] Connection header received
0808 17:01:35.880349 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:35.880350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:35.880362 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:35.880424 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:35.880842 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:35.881609 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:35.882787 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:35.883160 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:35.883219 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:35.885342 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:35.885353 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:35.885354 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:35.885365 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:35.887127 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:35.887487 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:35.887491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:35.887491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:35.887548 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:35.887751 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:35.887752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:35.887753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:35.887908 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:35.888006 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:35.888121 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 17:01:35.888248 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
I20260808 17:01:35.889015 9605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:01:35.889458 9609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.889290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:42869 (local address 127.0.0.1:49028)
0808 17:01:35.889349 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.889408 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 17:01:35.889621 9528 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.14.124:42869: connect: Connection refused (error 111)
I20260808 17:01:35.890079 9609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.889961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:42869 (local address 127.0.0.1:49038)
0808 17:01:35.890031 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.890056 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 17:01:35.890192 9605 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.124:42869: Network error: Client connection negotiation failed: client connection to 127.8.14.124:42869: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.14.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.124.keytab.
Entry for principal kudu/127.8.14.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.124.keytab.
I20260808 17:01:35.906050 8249 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskiOuv3j/build/release/bin/kudu
/tmp/dist-test-taskiOuv3j/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.124:42869
--webserver_interface=127.8.14.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.124.keytab
--principal=kudu/127.8.14.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.14.84:46269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.14.126:34873,127.8.14.125:46603,127.8.14.124:42869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 17:01:35.937729 9495 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } attempt: 3
I20260808 17:01:35.939412 9605 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } attempt: 1
I20260808 17:01:35.939815 9609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.939689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:42869 (local address 127.0.0.1:49044)
0808 17:01:35.939750 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.939783 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 17:01:35.940201 9609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.940101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:42869 (local address 127.0.0.1:49054)
0808 17:01:35.940151 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.940177 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 17:01:35.940385 9605 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.124:42869: Network error: Client connection negotiation failed: client connection to 127.8.14.124:42869: connect: Connection refused (error 111)
Aug 08 17:01:35 dist-test-slave-1nl6 krb5kdc[9384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208495, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for kudu/127.8.14.125@KRBTEST.COM
I20260808 17:01:35.945395 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.938125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:46603 (local address 127.0.0.1:43386)
0808 17:01:35.938189 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.938206 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:35.938292 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:35.938447 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:35.938448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:35.938615 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:35.938678 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:35.938680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:35.939489 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:35.939491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:35.940329 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:35.940332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:35.940888 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:35.940890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:35.940960 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:35.941441 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:35.941445 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:35.942666 (+ 1221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:35.944394 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:35.944395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:35.944395 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:35.944564 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:35.944800 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:35.944801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:35.944801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:35.944853 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:35.945164 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:35.945165 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:35.945204 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 17:01:35.945257 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 17:01:35.945332 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 17:01:35.945525 9612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.938209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43386 (local address 127.8.14.125:46603)
0808 17:01:35.938303 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:35.938306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:35.938317 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:01:35.938359 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:35.938360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:35.938372 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:35.938439 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:35.938785 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:35.939393 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:35.940442 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:35.940806 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:35.940880 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:35.942751 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:35.942753 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:35.942753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:35.942766 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:35.944308 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:35.944646 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:35.944648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:35.944649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:35.944721 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:35.944923 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:35.944924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:35.944924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:35.945089 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:35.945174 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:35.945313 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 17:01:35.945408 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 17:01:35.946075 9495 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:01:35.946424 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.946314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:42869 (local address 127.0.0.1:49066)
0808 17:01:35.946366 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.946393 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 17:01:35.946787 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.946687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:42869 (local address 127.0.0.1:49078)
0808 17:01:35.946737 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.946772 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 17:01:35.946914 9495 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.124:42869: Network error: Client connection negotiation failed: client connection to 127.8.14.124:42869: connect: Connection refused (error 111)
I20260808 17:01:35.978046 9495 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } attempt: 1
I20260808 17:01:35.978582 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.978416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:42869 (local address 127.0.0.1:49092)
0808 17:01:35.978499 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.978546 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 17:01:35.979058 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:35.978917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:42869 (local address 127.0.0.1:49100)
0808 17:01:35.978994 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:35.979029 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 17:01:35.979173 9495 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.124:42869: Network error: Client connection negotiation failed: client connection to 127.8.14.124:42869: connect: Connection refused (error 111)
W20260808 17:01:35.992399 9616 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 17:01:35.992626 9616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 17:01:35.992663 9616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 17:01:35.992695 9616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 17:01:35.993765 9616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 17:01:35.993832 9616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 17:01:35.993857 9616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 17:01:35.993885 9616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 17:01:35.993904 9616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 17:01:35.995440 9616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.14.84:46269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.14.126:34873,127.8.14.125:46603,127.8.14.124:42869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/krb5kdc/kudu_127.8.14.124.keytab
--principal=kudu/127.8.14.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.14.124:42869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.14.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 16:05:46 UTC on ccycloud.quasar-ybyrao.root.comops.site
build id 4575301
I20260808 17:01:35.995704 9616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 17:01:35.995932 9616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 17:01:36.000855 9622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 17:01:36.000897 9624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:36.000914 9616 server_base.cc:1034] running on GCE node
W20260808 17:01:36.000887 9627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 17:01:36.001346 9616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 17:01:36.001675 9616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 17:01:36.002825 9616 hybrid_clock.cc:648] HybridClock initialized: now 1786208496002811 us; error 28 us; skew 500 ppm
Aug 08 17:01:36 dist-test-slave-1nl6 krb5kdc[9384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208496, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 17:01:36.005281 9616 init.cc:377] Logged in from keytab as kudu/127.8.14.124@KRBTEST.COM (short username kudu)
I20260808 17:01:36.005800 9616 webserver.cc:460] Webserver started at http://127.8.14.124:42303/ using document root <none> and password file <none>
I20260808 17:01:36.006064 9616 fs_manager.cc:359] Metadata directory not provided
I20260808 17:01:36.006138 9616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 17:01:36.006302 9616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 17:01:36.007417 9616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-2/data/instance:
uuid: "b873ad6c7f454dde9b27fbd67e99bb07"
format_stamp: "Formatted at 2026-08-08 17:01:36 on dist-test-slave-1nl6"
server_key: "aa8f9ca02fbd086d36a356d7ab304abf"
server_key_iv: "38ce7509fc95b97467a281d1f0b9262a"
server_key_version: "encryptionkey@0"
I20260808 17:01:36.007779 9616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-2/wal/instance:
uuid: "b873ad6c7f454dde9b27fbd67e99bb07"
format_stamp: "Formatted at 2026-08-08 17:01:36 on dist-test-slave-1nl6"
server_key: "aa8f9ca02fbd086d36a356d7ab304abf"
server_key_iv: "38ce7509fc95b97467a281d1f0b9262a"
server_key_version: "encryptionkey@0"
I20260808 17:01:36.008525 9605 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } attempt: 2
I20260808 17:01:36.008890 9609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.008766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:42869 (local address 127.0.0.1:49104)
0808 17:01:36.008830 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:36.008863 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 17:01:36.009262 9616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 17:01:36.009317 9609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.009201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:42869 (local address 127.0.0.1:49110)
0808 17:01:36.009257 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:36.009284 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.14.124:42869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 17:01:36.009485 9605 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.14.124:42869: Network error: Client connection negotiation failed: client connection to 127.8.14.124:42869: connect: Connection refused (error 111)
I20260808 17:01:36.010084 9635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 17:01:36.010269 9616 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 17:01:36.010375 9616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-2/data,/tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-2/wal
uuid: "b873ad6c7f454dde9b27fbd67e99bb07"
format_stamp: "Formatted at 2026-08-08 17:01:36 on dist-test-slave-1nl6"
server_key: "aa8f9ca02fbd086d36a356d7ab304abf"
server_key_iv: "38ce7509fc95b97467a281d1f0b9262a"
server_key_version: "encryptionkey@0"
I20260808 17:01:36.010459 9616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 17:01:36.029827 9616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 17:01:36.032264 9616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 17:01:36.032434 9616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 17:01:36.036334 9616 rpc_server.cc:307] RPC server started. Bound to: 127.8.14.124:42869
I20260808 17:01:36.036388 9718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.14.124:42869 every 8 connection(s)
I20260808 17:01:36.036818 9616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-2/data/info.pb
I20260808 17:01:36.037853 9722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 17:01:36.039322 9722 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:01:36.042054 8249 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskiOuv3j/build/release/bin/kudu as pid 9616
I20260808 17:01:36.042177 8249 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskiOuv3j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786208190438830-8249-0/minicluster-data/master-2/wal/instance
I20260808 17:01:36.042420 8249 external_mini_cluster.cc:1428] Setting key 80a5b68a059722471c897cfd811a6095
Aug 08 17:01:36 dist-test-slave-1nl6 krb5kdc[9384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208496, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.124@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
Aug 08 17:01:36 dist-test-slave-1nl6 krb5kdc[9384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.124@KRBTEST.COM
I20260808 17:01:36.049429 9498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.039996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39776 (local address 127.8.14.126:34873)
0808 17:01:36.040093 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.040096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.040259 (+ 163us) server_negotiation.cc:407] Connection header received
0808 17:01:36.040776 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.040777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.040790 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.040865 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.041490 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.041992 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.043569 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.043781 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.043894 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:36.046228 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:36.046230 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:36.046231 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:36.046238 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:36.047669 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.048177 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.048179 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.048179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.048240 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.048608 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.048610 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.048610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.048780 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:36.048854 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:36.049221 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 17:01:36.049316 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 17:01:36.049597 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.039918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.126:34873 (local address 127.0.0.1:39776)
0808 17:01:36.040134 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:36.040175 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:36.040663 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:36.040959 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:36.040962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:36.041192 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:36.041385 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:36.041389 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.042099 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.042100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:36.043470 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:36.043472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.043917 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.043919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:36.044066 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:36.044597 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:36.044617 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:36.046129 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:36.047776 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:36.047784 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:36.047787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:36.048072 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:36.048336 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:36.048337 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:36.048337 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:36.048507 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:36.048859 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:36.048867 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:36.049036 (+ 169us) client_negotiation.cc:769] Sending connection context
0808 17:01:36.049178 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 17:01:36.049346 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":86,"threads_started":1}
I20260808 17:01:36.050179 9722 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:01:36.051507 9726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.042841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49118 (local address 127.8.14.124:42869)
0808 17:01:36.043045 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.043050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.043210 (+ 160us) server_negotiation.cc:407] Connection header received
0808 17:01:36.043240 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.043242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.043269 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.043359 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.043937 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.044623 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.045365 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.045714 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.045799 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:36.048314 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:36.048323 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:36.048323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:36.048330 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:36.050235 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.050619 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.050622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.050622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.050690 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.050951 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.050952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.050953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.051122 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:36.051283 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:36.051327 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 17:01:36.051404 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":119,"threads_started":1}
I20260808 17:01:36.054320 9495 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } attempt: 2
Aug 08 17:01:36 dist-test-slave-1nl6 krb5kdc[9384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208496, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.124@KRBTEST.COM for kudu/127.8.14.125@KRBTEST.COM
Aug 08 17:01:36 dist-test-slave-1nl6 krb5kdc[9384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.126@KRBTEST.COM
Aug 08 17:01:36 dist-test-slave-1nl6 krb5kdc[9384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208495, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.126@KRBTEST.COM for kudu/127.8.14.124@KRBTEST.COM
I20260808 17:01:36.059469 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.050468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.125:46603 (local address 127.0.0.1:43398)
0808 17:01:36.050530 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:36.050541 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:36.050584 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:36.050755 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:36.050757 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:36.050905 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:36.050958 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:36.050959 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.051808 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.051809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:36.052569 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:36.052572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.053064 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.053065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:36.053152 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:36.053603 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:36.053608 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:36.054952 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:36.058065 (+ 3113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:36.058066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:36.058066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:36.058284 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:36.058535 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:36.058535 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:36.058536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:36.058600 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:36.059089 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:36.059090 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:36.059136 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 17:01:36.059213 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 17:01:36.059348 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 17:01:36.059746 9612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.050526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43398 (local address 127.8.14.125:46603)
0808 17:01:36.050568 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.050570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.050580 (+ 10us) server_negotiation.cc:407] Connection header received
0808 17:01:36.050649 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.050651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.050662 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.050733 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.051019 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.051724 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.052665 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.052983 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.053034 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:36.055062 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:36.055064 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:36.055064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:36.055072 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:36.057339 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.058377 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.058378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.058378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.058442 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.058806 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.058807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.058808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.058987 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:36.059073 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:36.059260 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 17:01:36.059343 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:01:36.060317 9722 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } has no permanent_uuid. Determining permanent_uuid...
I20260808 17:01:36.061599 9498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.052701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39786 (local address 127.8.14.126:34873)
0808 17:01:36.052765 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.052766 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.052773 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:01:36.052791 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.052792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.052800 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.052853 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.053435 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.053872 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.054567 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.054934 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.054992 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:36.057621 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:36.057622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:36.057623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:36.057628 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:36.059926 (+ 2298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.060343 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.060345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.060345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.060472 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.060878 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.060879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.060880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.061019 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:36.061198 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:36.061403 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 17:01:36.061479 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 17:01:36.062371 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.054565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:42869 (local address 127.0.0.1:49128)
0808 17:01:36.054656 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:36.054666 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:36.054704 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:36.054839 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:36.054839 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:36.055027 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:36.055070 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:36.055072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.055813 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.055814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:36.056487 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:36.056491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.056967 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.056968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:36.057026 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:36.057429 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:36.057436 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:36.059111 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:36.061070 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:36.061071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:36.061071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:36.061315 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:36.061524 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:36.061524 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:36.061525 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:36.061583 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:36.062035 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:36.062036 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:36.062085 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 17:01:36.062168 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 17:01:36.062277 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 17:01:36.062680 9726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.054653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49128 (local address 127.8.14.124:42869)
0808 17:01:36.054692 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.054694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.054702 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:01:36.054773 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.054773 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.054783 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.054834 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.055166 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.055714 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.056583 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.056859 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.056965 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:36.059203 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:36.059204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:36.059205 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:36.059210 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:36.060982 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.061397 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.061398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.061398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.061452 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.061832 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.061833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.061834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.061945 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:36.062037 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:36.062498 (+ 461us) server_negotiation.cc:299] Negotiation successful
0808 17:01:36.062580 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 17:01:36.064574 9495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5: Bootstrap starting.
Aug 08 17:01:36 dist-test-slave-1nl6 krb5kdc[9384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208496, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.124@KRBTEST.COM for kudu/127.8.14.124@KRBTEST.COM
I20260808 17:01:36.065505 9495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:36.065877 9495 log.cc:824] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:36.067193 9495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5: No bootstrap required, opened a new log
Aug 08 17:01:36 dist-test-slave-1nl6 krb5kdc[9384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.14.125@KRBTEST.COM
I20260808 17:01:36.068909 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.060581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:42869 (local address 127.0.0.1:49130)
0808 17:01:36.060631 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:36.060641 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:36.060679 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:36.060930 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:36.060931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:36.061096 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:36.061138 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:36.061140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.062079 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.062080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:36.062991 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:36.062994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.063949 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.063950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:36.063998 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:36.064414 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:36.064418 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:36.065771 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:36.067781 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:36.067782 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:36.067783 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:36.067997 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:36.068201 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:36.068202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:36.068202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:36.068272 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:36.068594 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:36.068595 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:36.068634 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 17:01:36.068704 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 17:01:36.068802 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 17:01:36.069221 9728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.060626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49130 (local address 127.8.14.124:42869)
0808 17:01:36.060809 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.060811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.060819 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:01:36.060839 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.060840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.060850 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.060924 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.061224 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.062005 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.063660 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.063852 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.063946 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:36.065871 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:36.065872 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:36.065873 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:36.065879 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:36.067679 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.068080 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.068081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.068081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.068141 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.068357 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.068358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.068358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.068507 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:36.068603 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:36.069035 (+ 432us) server_negotiation.cc:299] Negotiation successful
0808 17:01:36.069109 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":88,"threads_started":1}
I20260808 17:01:36.069751 9495 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } }
I20260808 17:01:36.069914 9495 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f8c02b96c34c93a397b17a27d21dc5, State: Initialized, Role: FOLLOWER
I20260808 17:01:36.070004 9495 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } }
I20260808 17:01:36.070299 9730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } } }
I20260808 17:01:36.070453 9730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:01:36.070494 9495 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:36.070725 9722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07: Bootstrap starting.
I20260808 17:01:36.070974 9612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.062231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43408 (local address 127.8.14.125:46603)
0808 17:01:36.062272 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.062274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.062281 (+ 7us) server_negotiation.cc:407] Connection header received
0808 17:01:36.062300 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.062301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.062309 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.062367 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.063052 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.063582 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.064934 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.065285 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.065358 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:36.067913 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:36.067915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:36.067915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:36.067922 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:36.069611 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.070018 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.070019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.070019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.070074 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.070278 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.070279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.070279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.070396 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:36.070529 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:36.070768 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 17:01:36.070842 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 17:01:36.071524 9722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:36.071807 9722 log.cc:824] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07: Log is configured to *not* fsync() on all Append() calls
W20260808 17:01:36.073060 9744 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:01:36.073114 9744 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:01:36.073786 9722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07: No bootstrap required, opened a new log
I20260808 17:01:36.075874 9722 raft_consensus.cc:348] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } }
I20260808 17:01:36.076021 9722 raft_consensus.cc:729] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b873ad6c7f454dde9b27fbd67e99bb07, State: Initialized, Role: FOLLOWER
I20260808 17:01:36.076103 9722 consensus_queue.cc:260] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } }
I20260808 17:01:36.073354 9400 master_runner.cc:186] Error getting master registration for 127.8.14.125:46603: OK, instance_id { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" instance_seqno: 1786208495864898 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 17:01:36.076272 9400 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 17:01:36.076416 9722 sys_catalog.cc:564] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:36.077716 8249 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 17:01:36.078701 9459 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "90f8c02b96c34c93a397b17a27d21dc5"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39786
I20260808 17:01:36.079080 9459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 17:01:36.079144 9459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:36.079877 9459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } }
I20260808 17:01:36.080157 9459 leader_election.cc:288] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [CANDIDATE]: Term 1 election: Requested vote from peers 61190ccb92714701a0a8c5081c03ab41 (127.8.14.125:46603), b873ad6c7f454dde9b27fbd67e99bb07 (127.8.14.124:42869)
I20260808 17:01:36.080499 9677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90f8c02b96c34c93a397b17a27d21dc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b873ad6c7f454dde9b27fbd67e99bb07"
I20260808 17:01:36.080607 9677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07 [term 0 FOLLOWER]: Advancing to term 1
W20260808 17:01:36.078126 9764 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:01:36.081350 9764 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:01:36.079150 9747 sys_catalog.cc:455] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } } }
I20260808 17:01:36.081422 9747 sys_catalog.cc:458] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:01:36.081478 9677 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f8c02b96c34c93a397b17a27d21dc5 in term 1.
I20260808 17:01:36.081670 9430 leader_election.cc:302] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f8c02b96c34c93a397b17a27d21dc5, b873ad6c7f454dde9b27fbd67e99bb07; no voters:
I20260808 17:01:36.081840 9730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 17:01:36.081987 9730 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [term 1 LEADER]: Becoming Leader. State: Replica: 90f8c02b96c34c93a397b17a27d21dc5, State: Running, Role: LEADER
I20260808 17:01:36.082082 9730 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } }
I20260808 17:01:36.082547 9730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90f8c02b96c34c93a397b17a27d21dc5. Latest consensus state: current_term: 1 leader_uuid: "90f8c02b96c34c93a397b17a27d21dc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } } }
I20260808 17:01:36.082646 9730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:36.082821 9781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 17:01:36.083058 9781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 17:01:36.083606 9616 master_runner.cc:186] Error getting master registration for 127.8.14.125:46603: OK, instance_id { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" instance_seqno: 1786208495864898 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 17:01:36.085737 9616 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 17:01:36.092870 9498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.082797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39800 (local address 127.8.14.126:34873)
0808 17:01:36.082833 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.082835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.084305 (+ 1470us) server_negotiation.cc:407] Connection header received
0808 17:01:36.084525 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.084526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.084536 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.084567 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.085395 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.086167 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.087287 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.087577 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.087618 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:36.089371 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:36.089372 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:36.089373 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:36.089380 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:36.091247 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.091782 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.091784 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.091784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.091849 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.092081 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.092082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.092082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.092247 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:36.092346 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:36.092661 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 17:01:36.092740 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:01:36.095112 9728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.081872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49134 (local address 127.8.14.124:42869)
0808 17:01:36.081912 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.081914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.084131 (+ 2217us) server_negotiation.cc:407] Connection header received
0808 17:01:36.084237 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.084238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.084254 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.084306 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.085648 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.086344 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.087739 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.088460 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.088507 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:36.091438 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:36.091439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:36.091440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:36.091447 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:36.093358 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.093862 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.093864 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.093864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.093922 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.094553 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.094554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.094554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.094667 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:36.094756 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:36.094964 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 17:01:36.095035 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 17:01:36.099728 9677 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07 [term 1 FOLLOWER]: Refusing update from remote peer 90f8c02b96c34c93a397b17a27d21dc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:36.100063 9774 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:36.100629 9605 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } attempt: 3
I20260808 17:01:36.101763 9747 sys_catalog.cc:455] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90f8c02b96c34c93a397b17a27d21dc5. Latest consensus state: current_term: 1 leader_uuid: "90f8c02b96c34c93a397b17a27d21dc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } } }
I20260808 17:01:36.101876 9747 sys_catalog.cc:458] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07 [sys.catalog]: This master's current role is: FOLLOWER
Aug 08 17:01:36 dist-test-slave-1nl6 krb5kdc[9384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786208495, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.14.125@KRBTEST.COM for kudu/127.8.14.124@KRBTEST.COM
I20260808 17:01:36.106593 9612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.082503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43412 (local address 127.8.14.125:46603)
0808 17:01:36.082542 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.082544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.084411 (+ 1867us) server_negotiation.cc:407] Connection header received
0808 17:01:36.084459 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.084460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.084470 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.084517 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.094416 (+ 9899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.095088 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.096535 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.096738 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.096831 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:36.098285 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:36.098286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:36.098287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:36.098293 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:36.100022 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.103975 (+ 3953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.103977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.103977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.104048 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.104366 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.104367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.104368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.104489 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:36.104567 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:36.105160 (+ 593us) server_negotiation.cc:299] Negotiation successful
0808 17:01:36.106492 (+ 1332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:01:36.106968 9774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "90f8c02b96c34c93a397b17a27d21dc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } } }
I20260808 17:01:36.107103 9774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [sys.catalog]: This master's current role is: LEADER
I20260808 17:01:36.107735 9747 sys_catalog.cc:455] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "90f8c02b96c34c93a397b17a27d21dc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } } }
I20260808 17:01:36.108078 9747 sys_catalog.cc:458] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:01:36.108681 9782 mvcc.cc:205] Tried to move back new op lower bound from 7316310000005795840 to 7316309999953002496. Current Snapshot: MvccSnapshot[applied={T|T < 7316310000005795840}]
I20260808 17:01:36.108843 9783 mvcc.cc:205] Tried to move back new op lower bound from 7316310000005795840 to 7316309999953002496. Current Snapshot: MvccSnapshot[applied={T|T < 7316310000005795840}]
I20260808 17:01:36.110702 9781 catalog_manager.cc:1347] Generated new cluster ID: 1da0796310e24ffc94a8811783f2758a
I20260808 17:01:36.110759 9781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 17:01:36.111946 9609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.101498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.14.124:42869 (local address 127.0.0.1:49146)
0808 17:01:36.101977 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 17:01:36.101988 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 17:01:36.102031 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 17:01:36.102256 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 17:01:36.102258 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 17:01:36.102405 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 17:01:36.102444 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:36.102445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.103336 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.103337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:36.104117 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 17:01:36.104119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.104658 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.104659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 17:01:36.104727 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:36.105145 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 17:01:36.105150 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 17:01:36.107498 (+ 2348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 17:01:36.109703 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:36.109705 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:36.109705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:36.109940 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:36.110317 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 17:01:36.110318 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 17:01:36.110318 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 17:01:36.110399 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 17:01:36.111634 (+ 1235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 17:01:36.111635 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 17:01:36.111702 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 17:01:36.111762 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 17:01:36.111853 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459}
I20260808 17:01:36.112555 9728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:36.102097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49146 (local address 127.8.14.124:42869)
0808 17:01:36.102146 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.102148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.102156 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:01:36.102176 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.102176 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.102185 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.102242 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.102521 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.103253 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.104212 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:36.104538 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:36.104596 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:36.107595 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 17:01:36.107596 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 17:01:36.107596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 17:01:36.107602 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 17:01:36.109583 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.110138 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.110140 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.110140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.110220 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 17:01:36.110480 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 17:01:36.110481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 17:01:36.110481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 17:01:36.110651 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 17:01:36.110727 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:36.112393 (+ 1666us) server_negotiation.cc:299] Negotiation successful
0808 17:01:36.112472 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 17:01:36.120410 9605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41: Bootstrap starting.
I20260808 17:01:36.121145 9605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41: Neither blocks nor log segments found. Creating new log.
I20260808 17:01:36.121413 9605 log.cc:824] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41: Log is configured to *not* fsync() on all Append() calls
I20260808 17:01:36.122465 9605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41: No bootstrap required, opened a new log
I20260808 17:01:36.124590 9781 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 17:01:36.124737 9605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } }
I20260808 17:01:36.124876 9605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61190ccb92714701a0a8c5081c03ab41, State: Initialized, Role: FOLLOWER
I20260808 17:01:36.124946 9605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } }
I20260808 17:01:36.125283 9605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 17:01:36.125304 9781 catalog_manager.cc:1504] Loading token signing keys...
I20260808 17:01:36.126662 9570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41 [term 0 FOLLOWER]: Advancing to term 1
I20260808 17:01:36.126525 9572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90f8c02b96c34c93a397b17a27d21dc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "61190ccb92714701a0a8c5081c03ab41"
I20260808 17:01:36.129057 9570 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41 [term 1 FOLLOWER]: Refusing update from remote peer 90f8c02b96c34c93a397b17a27d21dc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 17:01:36.129087 9781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5: Generated new TSK 0
I20260808 17:01:36.129349 9730 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 17:01:36.129886 9781 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 17:01:36.130312 9572 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90f8c02b96c34c93a397b17a27d21dc5 in term 1.
W20260808 17:01:36.131095 9803 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 17:01:36.131151 9803 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 17:01:36.132794 9790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90f8c02b96c34c93a397b17a27d21dc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } } }
I20260808 17:01:36.132957 9790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:01:36.133054 9790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90f8c02b96c34c93a397b17a27d21dc5. Latest consensus state: current_term: 1 leader_uuid: "90f8c02b96c34c93a397b17a27d21dc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } } }
I20260808 17:01:36.133153 9790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:01:36.133229 9790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "90f8c02b96c34c93a397b17a27d21dc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } } }
I20260808 17:01:36.133325 9790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 17:01:36.135509 9806 mvcc.cc:205] Tried to move back new op lower bound from 7316310000138559488 to 7316309999953002496. Current Snapshot: MvccSnapshot[applied={T|T < 7316310000005795840}]
W20260808 17:01:36.169085 9498 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:36.167253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39804 (local address 127.8.14.126:34873)
0808 17:01:36.167302 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.167304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.168281 (+ 977us) server_negotiation.cc:407] Connection header received
0808 17:01:36.168404 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.168405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.168433 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.168464 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.169010 (+ 546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39804: BlockingRecv error: recv got EOF from 127.0.0.1:39804 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 17:01:36.169793 9728 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:36.167532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49162 (local address 127.8.14.124:42869)
0808 17:01:36.167570 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.167572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.168573 (+ 1001us) server_negotiation.cc:407] Connection header received
0808 17:01:36.169322 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.169323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.169334 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.169364 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.169731 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49162: BlockingRecv error: recv got EOF from 127.0.0.1:49162 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 17:01:36.170610 9612 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:36.167927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43416 (local address 127.8.14.125:46603)
0808 17:01:36.167964 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.167966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.170025 (+ 2059us) server_negotiation.cc:407] Connection header received
0808 17:01:36.170208 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.170209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.170220 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.170251 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.170553 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43416: BlockingRecv error: recv got EOF from 127.0.0.1:43416 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 17:01:36.176327 9612 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:36.175185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43420 (local address 127.8.14.125:46603)
0808 17:01:36.175252 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.175254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.175557 (+ 303us) server_negotiation.cc:407] Connection header received
0808 17:01:36.175719 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.175720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.175738 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.175795 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.176258 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43420: BlockingRecv error: recv got EOF from 127.0.0.1:43420 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 17:01:36.179317 9612 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:36.178536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43432 (local address 127.8.14.125:46603)
0808 17:01:36.178569 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.178570 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.178804 (+ 234us) server_negotiation.cc:407] Connection header received
0808 17:01:36.178892 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.178893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.178924 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.178953 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.179275 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43432: BlockingRecv error: recv got EOF from 127.0.0.1:43432 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 17:01:36.183220 9612 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:36.182392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43446 (local address 127.8.14.125:46603)
0808 17:01:36.182424 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.182426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.182689 (+ 263us) server_negotiation.cc:407] Connection header received
0808 17:01:36.182780 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.182781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.182790 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.182817 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.183169 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43446: BlockingRecv error: recv got EOF from 127.0.0.1:43446 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 17:01:36.189002 9612 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:36.188250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43458 (local address 127.8.14.125:46603)
0808 17:01:36.188283 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.188284 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.188547 (+ 263us) server_negotiation.cc:407] Connection header received
0808 17:01:36.188636 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.188637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.188647 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.188674 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.188961 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43458: BlockingRecv error: recv got EOF from 127.0.0.1:43458 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 17:01:36.198931 9612 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:36.198144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43464 (local address 127.8.14.125:46603)
0808 17:01:36.198206 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.198209 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.198426 (+ 217us) server_negotiation.cc:407] Connection header received
0808 17:01:36.198518 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.198519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.198528 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.198557 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.198863 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43464: BlockingRecv error: recv got EOF from 127.0.0.1:43464 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 17:01:36.217028 9612 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:36.216106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43472 (local address 127.8.14.125:46603)
0808 17:01:36.216145 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.216147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.216469 (+ 322us) server_negotiation.cc:407] Connection header received
0808 17:01:36.216574 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.216575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.216585 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.216614 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.216975 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43472: BlockingRecv error: recv got EOF from 127.0.0.1:43472 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 17:01:36.251495 9612 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:36.250476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43488 (local address 127.8.14.125:46603)
0808 17:01:36.250522 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.250525 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.250820 (+ 295us) server_negotiation.cc:407] Connection header received
0808 17:01:36.250958 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.250959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.250971 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.251002 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.251430 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43488: BlockingRecv error: recv got EOF from 127.0.0.1:43488 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 17:01:36.317797 9612 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:36.316991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43492 (local address 127.8.14.125:46603)
0808 17:01:36.317079 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.317082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.317182 (+ 100us) server_negotiation.cc:407] Connection header received
0808 17:01:36.317276 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.317277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.317290 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.317354 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.317721 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43492: BlockingRecv error: recv got EOF from 127.0.0.1:43492 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 17:01:36.447949 9612 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:36.447092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43504 (local address 127.8.14.125:46603)
0808 17:01:36.447201 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.447204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.447317 (+ 113us) server_negotiation.cc:407] Connection header received
0808 17:01:36.447435 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.447436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.447449 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.447520 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.447879 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43504: BlockingRecv error: recv got EOF from 127.0.0.1:43504 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 17:01:36.508364 9730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "90f8c02b96c34c93a397b17a27d21dc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f8c02b96c34c93a397b17a27d21dc5" member_type: VOTER last_known_addr { host: "127.8.14.126" port: 34873 } } peers { permanent_uuid: "61190ccb92714701a0a8c5081c03ab41" member_type: VOTER last_known_addr { host: "127.8.14.125" port: 46603 } } peers { permanent_uuid: "b873ad6c7f454dde9b27fbd67e99bb07" member_type: VOTER last_known_addr { host: "127.8.14.124" port: 42869 } } }
I20260808 17:01:36.508617 9730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90f8c02b96c34c93a397b17a27d21dc5 [sys.catalog]: This master's current role is: LEADER
W20260808 17:01:36.705988 9612 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 17:01:36.705250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43516 (local address 127.8.14.125:46603)
0808 17:01:36.705336 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:36.705338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:36.705349 (+ 11us) server_negotiation.cc:407] Connection header received
0808 17:01:36.705455 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:36.705456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:36.705470 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:36.705551 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 17:01:36.705926 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43516: BlockingRecv error: recv got EOF from 127.0.0.1:43516 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 17:01:37.081791 9764 catalog_manager.cc:1259] Loaded cluster ID: 1da0796310e24ffc94a8811783f2758a
I20260808 17:01:37.081866 9764 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07: loading cluster ID for follower catalog manager: success
I20260808 17:01:37.082607 9764 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07: acquiring CA information for follower catalog manager: success
I20260808 17:01:37.082871 9764 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b873ad6c7f454dde9b27fbd67e99bb07: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 17:01:37.131569 9803 catalog_manager.cc:1259] Loaded cluster ID: 1da0796310e24ffc94a8811783f2758a
I20260808 17:01:37.131647 9803 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41: loading cluster ID for follower catalog manager: success
I20260808 17:01:37.132423 9803 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41: acquiring CA information for follower catalog manager: success
I20260808 17:01:37.132676 9803 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 61190ccb92714701a0a8c5081c03ab41: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 17:01:37.222240 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:37.219360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43528 (local address 127.8.14.125:46603)
0808 17:01:37.219500 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:37.219502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:37.219588 (+ 86us) server_negotiation.cc:407] Connection header received
0808 17:01:37.219698 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:37.219699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:37.219781 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:37.219876 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:37.220062 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:37.220572 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:37.221372 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:37.221598 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:37.221685 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:37.221870 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:37.221991 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:37.222090 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:37.222121 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 17:01:37.222141 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":57,"threads_started":1}
I20260808 17:01:37.226486 9917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:37.224095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49170 (local address 127.8.14.124:42869)
0808 17:01:37.224231 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:37.224234 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:37.224242 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:01:37.224279 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:37.224279 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:37.224319 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:37.224388 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:37.224500 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:37.224962 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:37.225675 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:37.225864 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:37.225915 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:37.226151 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:37.226253 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:37.226320 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:37.226360 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 17:01:37.226376 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
I20260808 17:01:37.231191 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:37.228476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43540 (local address 127.8.14.125:46603)
0808 17:01:37.228515 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:37.228517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:37.228545 (+ 28us) server_negotiation.cc:407] Connection header received
0808 17:01:37.228634 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:37.228635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:37.228646 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:37.228675 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:37.228884 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:37.229595 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:37.230335 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:37.230607 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:37.230667 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:37.230890 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:37.230941 (+ 51us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:37.230996 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:37.231079 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 17:01:37.231120 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 17:01:37.231483 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:37.228268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39806 (local address 127.8.14.126:34873)
0808 17:01:37.228381 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:37.228383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:37.228391 (+ 8us) server_negotiation.cc:407] Connection header received
0808 17:01:37.228438 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:37.228439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:37.228492 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:37.228716 (+ 224us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:37.228759 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:37.229284 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:37.230351 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:37.230689 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:37.230754 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:37.230951 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:37.231050 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:37.231283 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:37.231310 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 17:01:37.231340 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":54,"threads_started":1}
I20260808 17:01:37.231587 9917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 17:01:37.228344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49182 (local address 127.8.14.124:42869)
0808 17:01:37.228378 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 17:01:37.228380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 17:01:37.228578 (+ 198us) server_negotiation.cc:407] Connection header received
0808 17:01:37.228637 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 17:01:37.228638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 17:01:37.228647 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 17:01:37.228698 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 17:01:37.228852 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:37.229564 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:37.230540 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 17:01:37.230839 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 17:01:37.230918 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 17:01:37.231096 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 17:01:37.231136 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 17:01:37.231167 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 17:01:37.231493 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 17:01:37.231518 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 17:01:37.233115 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 9400
I20260808 17:01:37.251013 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 9502
I20260808 17:01:37.258893 8249 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskiOuv3j/build/release/bin/kudu with pid 9616
2026-08-08T17:01:37Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3004 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6026 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (306832 ms total)
[ PASSED ] 54 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
2 FAILED TESTS